We use llmlite, so we have centralized api key access and each user gets an allowance of usage per month. We provide guidance on what model to use went and what it's good for. We also have a small cluster of 7900xtx for local inference when you go over your allowance but itvonky supports low volume.

My company uses flutter as it's framework and our site with fonts is over 20mb; trying to do cache invalidation on the client is a nightmare. Our devs say there's nothing they can do to really reduce the package size as the majority of the dependencies are from our vendors software development kits. The bloat is amazing, devs see no reason to optimize as everyone is on 4G/LTE mobile or 100mbit connections so data transfer isn't a concern. I'm glad we're starting to reach a point where clients are starting to choak, it might cause these framework devs to release optimized code to remain relevant.

See: https://gisanddata.maps.arcgis... Infection rates per day are down, recovery rates are outpacing infection rates and fatality looks to be 3.6% currently (or there abouts). If China would have enacted measures more quickly things wouldn't have blown up so badly, most of the world is in fine shape. Anything that depends on China manufacturing however is another story. With a month of China being on lockdown, that will impact global pricing for sure. So infection scaring is a bit overblown but the global financial impact is going to be large.