AI agents fail to navigate Ruby codebases despite code generation capability

According to a benchmark study on GitHub, AI agents can generate Ruby code but lack ability to reliably find dependencies and execute changes across 13 real codebases using 5 different models. On small, readable libraries, agents found sufficient dependencies; on production applications, agents found only a fraction of dependents and incorrectly declared audits complete. Providing agents with structural codebase maps improved performance, with the strongest model gaining +0.26 mean recall.