Bito Reviews

Summary: Within a couple of weeks of usage, I had determined that Bito did not actually provide any real improvements over existing processes. Most of the losses are due to the fact that you cannot trust it to do what it is told, resulting in the need to perform the more manual processes anyway. If you cannot trust in a process being done as requested, and cannot trust the organization behind the process, then you are adding additional overhead instead of automation. I found the code completion of Tabnine, which is based on the content of your solution as a whole, to be far more productive than Bito. Even if it isn't correct 100% of the time, it is often significantly faster to review what it suggests and select it or continue on typing, than to retype entire blocks that you or others had entered for similar processes earlier.

Positive: * I liked that I would be able to see how much each member of the dev team was making use of the product, and could impose spending limits.
* I also liked that it would generate all unit tests for a process, rather than the user having to copy and paste or insert generated content into a test file.
* I was excited about having it perform code reviews, thinking that it would reduce the workload of Tech Leads.

Negative: * As you use the product for a while, you being finding that the initial estimations of efficiency are exaggerated. The work it performs is incomplete.
* You can highlight a section of code, such as the full content of a class, and tell it to perform a code review, but it may choose to perform a code review on your entire codebase instead, and it is very slow at perform said reviews.
* Code reviews are incomplete and of such poor quality, that you must still perform a full code review manually. Many of its recommendations are not completely accurate, and/or of no consequence.
* SURPRISE BILLING: You can set billing limits, but by design, the system will change those limits as it sees fit. I set an overage limit of $30 per user, which Bito recommended, but after running a couple of code reviews in which the system chose to review the entire codebase, the system automatically changed the limit to a value of $280 per user. There are many terms to describe this illegal behavior, such as bait-and-switch or false advertising, but regardless of what you call it, it is unfair, unethical, and illegal. When I contacted their support team about the issue, their response was, "Hi, for users who are using our Code Review agent our system changes the overage limit so that you don't hit the limits after a small number of code reviews." Had I not noticed the email sent after the fact, I could be spending a LOT more than I agreed to.
* Bito also does a very bad job of generating unit tests. After running a code coverage report, I found that it omitted entire methods. The tests that it generates for TypeScript in Jest, required a lot of additional work before they would run. The tests are poorly written, use too many mocked processes, fail to monitor spies for proper error messages, and require a lot more work than other test generation AIs.
* Bito also does not have very good code completion support in VS Code. The free version of Tabnine is far better.