If you have a strategy that would work better, by all means go talk to the Android security team about it. They'd be happy to pay you a lot of money, if your ideas are good, and feasible. Honestly, that's very unlikely; a large number of very smart people have been thinking hard about this since about 2013, but maybe you can see something no one else has.

The strategy has a few parts:

1. Scan the the Play store apps for malicious code using both static and dynamic analysis. This will never catch everything; it's a continual arms race and the malware authors have insuperable advantages, but what the scanning can do is make the malware authors have to work hard, raise their development costs.
2. Require app authors to identify themselves and pay a small fee. This is another "cost-increasing" strategy, one that is obviously imperfect, but surprisingly effective anyway, which is why they want to apply it to all apps, not just Play store apps.
3. Put some friction in place to make it somewhat difficult for clueless users to be tricked into installing software from outside the Play store. It would be far, far easier to simply block sideloading or third-party app stores, but Google is committed, philosophically, to keeping those options open (which is also why Pixel phones have unlocked bootloaders, Chromebooks have "dev move", etc.). This is a balancing act; it needs to be hard enough to prevent casual installation, but still accessible.
4. Google Play Protect scans all apps (regardless of installation source) on all GMS devices (devices with Google Play), with user permission. If known malware is identified, GPP can either warn the user or simply remove the app. AFAIK, the "remove the app" option has never been used, but the last time I asked about it was several years ago, so I could be wrong.
5. Safe by default APIs. APIs are designed to limit potential abuse. This can't always work, in some cases because necessary APIs are abusable and in some cases because APIs were deployed that no one realized could be abused, and taking APIs away is hard (breaks apps). 6. Tight app sandboxing and tight mandatory access control (SELinux). This is largely successful, most malware works by abusing legitimate APIs, not by breaking out of the app restrictions, but there are always exceptions.

Again, if you have better ideas, by all means share!

I didn't write that very clearly. Google doesn't have a problem with non-Play and sideloaded apps being installed in general. They do have a problem with malicious apps, regardless of source.

No... I'm pretty sure it's not that. Android does protect against malicious non-Play and sideloaded apps, and Google doesn't have a problem with those being installed. (Context: From 2014 until late last year I worked on the Android security team. I have pretty good knowledge of what they do and don't care about.)

I'd like to understand exactly what this part of the Google response is talking about. Maybe it's just throwaway FUD, not referring to anything in particular, but it could also be something the EU is demanding that is actually bad for users.

In my experience, that's Rust. Or maybe Haskell. The more guardrails the AI has, the better.

16kloc is a small project.

I should soften what I said before: You can get away with seat-of-the-pants work if one or more of: (a) your projects are small, (b) your projects are non-critical or (c) you invest in massive testing and validation.

In my old job, if I made one kind of mistake it could prevent three billion Android devices from being able to boot. If I made a different kind, it could result in the same number of devices being insecure against data extraction and other attacks (of course, OEMs often made those mistakes for me). In my new job, if I make mistakes vehicles can crash and people can die.

I want all the help the tooling can give me, and I spend a fair amount of my time building additional tools to prevent mistakes. None of it eliminates the need for testing, of course, especially when lives are on the line, but bugs that get caught by the compiler cost almost nothing. Bugs that get caught by QA, or, far worse, in the field, are very expensive.

Note that there are big systems built in Python. Instagram has millions of lines of Python code. They also mandate explicit type declarations everywhere and they built their own static typechecker, Pyre. Lots of other people have built typechecking libraries and tools for Python, trying to fix that problem. Maybe the next time I write Python I should try some of those tools.

Though I probably won't ever write any Python again. I really only use it for quick and dirty scripts, and these days I tell AI to write those for me, and I don't even bother to read the code. If it works, fine. If it doesn't, I tell the LLM to figure out why, and fix it.

Sure there are. Some bug reports, you look at them and dig in and say "Damn, that's obscure. It depends on subtly-wrong but arguably-correct-in-isolation mistakes in four different places. How the hell did they even find that?".

Granted that it's more common to glance at it and just be ashamed, or to look at it and say "Really? You think an attacker can do A, B, C, D, and E, all at the same time? Oh, and to do B or D they'd already have to have the system mostly compromised. Okay, I'll fix it, but no way in hell is that actually exploitable."

Sorry, but you're wrong.

Maybe... if the projections are in formal filings I don't think they can do that without risking SEC penalties. They can be optimistic about revenues or falling costs or things, because who can tell? But straight up ignoring things, probably not.

You said that this creates an arms race, which implies that it's a question of capability.

I strongly dislike dynamically-typed languages like Python. They're okay for toy programs but for anything of any size... you'd better have one hell of a good test suite because the tools give you absolutely no help. I want strong, static typing. I want a very picky compiler that won't accept anything that isn't exactly correct. C++ is good. Rust is better. I do like type inference so I don't have to manually specify types very often. C++ is reasonably good at this. Rust is better.

I also do a fair amount of work on very tiny systems, programming on the bare metal. Something like Python is a complete non-starter there, it just won't fit. C is the norm for those cases, but C is almost as loosey-goosey as Python, but without any run-time checks. C++ is much better than C. Rust is better yet.

The worst non-hardware bug I've ever dealt with in nearly 40 years of professional programming was in Python, using the Twisted framework. I traced it down to one place that took a pure abstract interface, instantiated it, then called methods on it!. Even logging the type of the instantiated object showed that it was the abstract type. Someone way too clever had built a dependency injection framework that was so magical that there was absolutely no way to figure out what dependency was being injected.

I also have AI write most of my code these days, but that actually increases my desire for a very picky compiler, because AI does a lot better with those guardrails in place.

That is completely and utterly irrelevant.

You need to read the story of the Paperclip Maximizer. It's an entertaining and humorous story, but the point is that instrumental goals are real and often diverge wildly from the intrinsic goals that drive them.

In this case, the LLM was directed to increase its CyberBench score, a sensible intrinsic goal for an AI being trained to be able to find vulns and write exploits. It chose to do that by breaking out of its container, hacking the company that creates the scores, stealing employee credentials, and breaking into the score database. Those intermediate steps were instrumental goals, and they obviously diverged wildly from the intended intrinsic goal.

Would you also have said "all it did was carry out its instructions" if it had found that in order to break in it needed to hire private soldiers to shoot their way in? Or take control of the US nuclear arsenal and issue a threat to HuggingFace that they increase its score or it would nuke them? Or...

Yes, these are fanciful scenarios. What they are not, however, is impossible scenarios. And as AI gets more and more capable, you really want them to be impossible. Alignment matters. If fast takeoff happens (that's a theorized situation where AI becomes capable of self-improvement and rapidly makes itself vastly smarter than any human), alignment may well be essential for human survival.

Both of those things are true, I'd guess.

I am not an accountant, but as I understand it, "Profit margin" is generally "operating profit margin", meaning revenues less operating expenses, all divided by revenues, and their operating expenses aren't that large, so they can absolutely have very strong operating profits. But they also have mind-bogglingly huge capital expenditures going on, so when you account for those they're losing a lot of money. I'm surprised they're projecting profitability as soon as 2030, actually. That's only 3.5 years away.

Well... an AI model that is capable of hacking into the benchmark database and altering its score is clearly better than one that is not.

Nope. You misunderstood. It's not that the US model wasn't able to detect it, it's that the US model's guardrails prevented it from explaining. This doesn't demonstrate a capability gap against Chinese models, it demonstrates a two-sided failure of the safety protections of the US models. On the one side, the safety guardrails on the OpenAI model failed to prevent the attack. On the other side the guardrails on whatever US model(s) they tried blocked the model(s) from explaining the attack (presumably to protect against the explained-to people from learning how to perform it).

It's absolutely nicer. I've been writing C++ for 36 years and really enjoy it, but I find Rust to be much more pleasant to work in.

I don't know about Qt from Rust (I avoid UI code). It's been 15 years since I wrote any Qt code so I don't really remember it all that well even from C++, and it's probably evolved, so I can't even guess about what it might be like and whether the impedance mismatches with Rust are significant -- The C++ and Rust object models are somewhat different, so there definitely can be mismatches.

But in general, I find Rust is just nicer to work in. I'm more productive and the code feels cleaner and tighter.

As for the "nagging"... that's really only a learning curve issue. It's a non-trivial learning curve, true, but once you get past it the borrow checker doesn't really get in your way. And what you get in trade for that learning curve is fantastic. You just don't have to think about a whole raft of memory-safety and concurrency problems, because the language guarantees they can't occur. You can get the memory safety guarantees in Python and similar language, but they come at a heavy runtime cost.

And, yes, I know all about "Modern C++" and how avoiding raw pointers, etc., is supposed to give you memory safety. And it does help, a lot... but you'd still better run valgrind on your programs to check for subtle mistakes, and even that might not catch them all.

Overall, Rust is just nicer.