Ethanol

That’s a great post, that well displays the issues with AI tests! For my own personal curiosity I looked at the testing rewrite of rsync, specifically the chgrp_test because it was the smallest test I quickly found. If you look at the original shell script, all it does is call chgrp and then fail if it doesn’t work. In the Python rewrite on the other hand the LLM calls chown to change the group and only if that fails, it tests chgrp. So if for some reason chown works but chgrp would fail, the original shell script would easily catch that (cause why do you test for chown anyways) while the Python rewrite doesn’t even call chgrp in case chown works.

Even though, this might not be as much of a problem in practice, I think it illustrates that the AI tends to write tests where it already anticipates and tries to fix potential issues, which absolutely goes against the use of tests!

This is actually quite fun and simple! Even if the problem and my following explanation look complicated :P

Let’s look at the three dimensional case. One can parametrize a 3 dimensional cube as the Cartesian product of intervals [0, 1] x [0, 1] x [0, 1]. This means a cube is a set of points (a, b, c) where a, b and c are real numbers between 0 and 1. The 2 dimensional sides of the cube are then given by fixing one coordinate. That is, the 6 sides are

{0}    x [0, 1] x [0, 1], 
{1}    x [0, 1] x [0, 1], 
[0, 1] x {0}    x [0, 1], 
[0, 1] x {1}    x [0, 1], 
[0, 1] x [0, 1] x {0} and 
[0, 1] x [0, 1] x {1}. 

Now we just start in the middle of a side at (0, 0.5, 0.5). To get to the next side we walk towards an edge (0, 0, 0.5) and then to the middle of the next side (0.5, 0, 0.5). We iterate this process until we run out of sides with a fixed 0, then walk towards a side with a fixed 1 and continue there. That is:

   (0  , 0.5, 0.5)
-> (0  , 0  , 0.5) 
-> (0.5, 0  , 0.5) 
-> (0.5, 0  , 0  ) 
-> (0.5, 0.5, 0  ) 
-> (1  , 0.5, 0  ) 
-> (1  , 0.5, 0.5) 
-> (1  , 1  , 0.5) 
-> (0.5, 1  , 0.5) 
-> (0.5, 1  , 1  ) 
-> (0.5, 0.5, 1  ) 

This path basically spirals around the cube, going through every side only once. Here’s a visualization (sorry, I’m no artist :P)

The same procedure works on a 4 dimensional cube or any other higher dimension. For the 4 dimensional cube it goes like this:

   (0  , 0.5, 0.5, 0.5)
-> (0  , 0  , 0.5, 0.5) 
-> (0.5, 0  , 0.5, 0.5) 
-> (0.5, 0  , 0  , 0.5) 
-> ...
-> (0.5, 0.5, 0.5, 0  )
-> (1  , 0.5, 0.5, 0  )
-> (1  , 0.5, 0.5, 0.5)
-> (1  , 1  , 0.5, 0.5)
-> ...
-> (0.5, 0.5, 0.5, 1  )

This works for arbitrary dimension except for the 1 dimensional cube (which is just a line) because the “sides” there are the two end points of the line and not connected at all. Additionally note, that it is never specified how edges count in this problem, whether they somehow count towards a face or whether you’re allowed to go back and fourth on edges. You could technically only walk along edges and step into the sides every now and then.