I get some people are afraid the world is going to shit, but policing thought isn't the solution. The solution is for everyone to distrust information and then spend time to dig deep. A fact isn't truth, it's information. How much information do you need to understand the problem varies depending on the topic

the downside of thought police and "fact checkers" is they fool themselves into thinking they know better. Everyone is different and keeping information from people is patronizing bullshit. Instead of shutting down people, the only real way to move forward is for everyone to talk to each other. Of course most people suck at listening and are locked into their own belief system, but policing thought just makes it worse.

of course Ackman is bitching and acting like a bully. Vampire capitalist don't actually give a shit about other people, they care about money, power, money, power and money. The more important question is this 'why do people in the US keep listening to dick head billionaires who have zero moral authority?"

I love how it's totally cool for OpenAI to use unlicensed data from the internet and not credit open source projects to build their own model. As soon as someone else does that to them, it's not right. What a load of hypocritical shit.

I contribute to open source and I don't mind OpenAI using it to train models. But just like a human, you better damn well cite and give credit to the projects.

you realize Intel is using TSMC now and they are still struggling after over a decade of bad CEO. The other big issue is the culture in the US. North american kids don't want to put in brutal hours in a chip fab. Who is going to run those fabs?

it's not that simple. The models memorize a significant amount of the data it was trained on. So it is violating copyright since it has a copy of the original artwork in the neural network weights. To get around this, the technology has to progress to the point where engineers can clearly prove the model didn't memorize the training data. We still can't interpret the weights. Don't take my word, go search arxiv.org for papers.

sounds like you have first hand experience with COBOL. I've seen mainframes apps in healthcare and the piles of ugly COBOL is deep and messy. I've forgotten how many times we were told "it does this" and we find out "nope it does something different."

then we spend a bunch of time documenting what it actually does and working around the issues we find.

that sounds reasonable on the surface, but it's actually not possible. The field of interpretability of models is an active area of research and no one has come up with a general purpose algorithm or theory. If you look at the research in that domain, you'll see that often researchers think the activations are doing a specific thing, but as they test they realize "nope, it's not doing that at all." Right now there's no clear way of deciphering the model weights (aka parameters) and accurately stating what it's doing in the model. Most of the research to date has shown we can't interpret the weights yet. Perhaps in the future, but right now very few people even have the skill to interpret a random model.

The secret sauce isn't how many layers and the sequence you order them. It's the weights produced by the training. You can run 12 separate training sessions with wildly different results.

it can't be really hard, Elon says it's easy and should be shipped by 2021

joking aside, even Karpathy said recently "FSD actually is harder than you think at first." Thing would be better if Elon shut his mouth and stop setting unreal expectations.

that one is both tragic and funny. I guess wet cement was an "out of distribution" problem that cruise didn't consider. The thing is, computer vision is powerful and cool, but sometimes pure vision alone isn't enough. No matter what Elon says, pure vision is not sufficient to understand (ie perceive) the environment. At minimum, there needs to be a higher level perception that understands "there's road work" and things to look out for.

exactly! that shit tastes terrible. the real issue is the sugar lobby has fucked up the US for the sake of profit. the country would be better off if we consumed 1/5th the sugar.

GPT doesn't crawl a lot of docs.

OpenAI collected several datasets, including common-crawl, reddit, wikipedia, github and other popular language datasets. They spent way more than 100 million to get to GPT4. They've been working on this for years and the training time is typically 6-8 months. The amount of electricity used to run 10K GPU for 8 months is crazy expensive. Training GPT3 was only made possible by NVidia DGX with 8 A100 cards. Before that, training GPT-3 175 billion parameter model wasn't really practical. It costs ALOT of money to build out a datacenter with 10K A100. Divide 10K / 8 A100 cards you get 1,250 DGX. Each DGX costs any where from 150-200K, if you use 150K the hardware alone costs 187 million.

Anyone watch the snowden movie? The US doesn't need some social media app, NSA just hacks all the backbone routers to full access to everyone's data.

on the otherhand, I'm glad china hasn't done what the NSA does, since I don't use TikTok. Not that it matters, since the NSA already has access to every citizen's data. So much for privacy.

I have some utilities for tensorflow written in python, but it so soooo painfully slow I reimplemented it in Java. I thought it might be 10-100x faster. After I rewrote the code, and benchmarked it, turns out it is 500x faster than python.

some things are good in python as long as you don't need performance. if memory and performance matters, write it in some other language. there are nice things about python, performance isn't one of them.

you mean like how Azure went down recentlly :) Keeping something like ChatGPT running smoothly and efficiently isn't an easy task. I've been doing cloud dev for over 10 years and I have painful memories of AWS and Azure going down. It's always fantastic to file a issue with AWS "hey this service is down", they respond with "no it's fine". Hours later news reports AWS is down and my ticket gets closed with no apology or acknowledgement "sorry we screwed up".

are you living in an alternate universe? Google wasn't first or even second. There was alta vista and yahoo. Keep in mind running chatGPT requires a shit ton of hardware and they are using Azure right now. Google has all of those TPU pods at their disposable to serve up chat style search. It's not like azure can magically get 1000 new racks up to handle chatGPT loads. I'm not a fan of google and they've acting like evil jerks lately, but they do have a huge hardware advantage.