Ant Developing Large Language Model Technology

Jack Ma-backed Ant Group is developing large-language model technology that will power ChatGPT-style services, joining a list of Chinese companies seeking to win an edge in next-generation artificial intelligence. From a report: The project known as "Zhen Yi" is being created by a dedicated unit and will deploy in-house research. An Ant spokesperson confirmed the news which was first reported by Chinastarmarket.cn Ant is racing against companies including its affiliate Alibaba Group Holding Ltd., Baidu and SenseTime. Their efforts mirror developments in US where Alphabet's Google and Microsoft are exploring generative AI, which can create original content from poetry to art just with simple user prompts.

Disappointment

[GlimmervoidG]

I thought this was about ants the insect, not a company. I am now very disappointed

Re:

[NobleNobbler]

Same. I thought our insectoid friends were finally breaking free of their oppressive caste system.

Re:

[antdude]

Me too! I was like huh? :O

Re:

[Motleypuss]

You and me both; what could an ant develop with enough stimulants and BCIs inserted into its tiny little body? Someone call the FTC on this farticle!

Soon, everyone will have AI

[Okian Warrior]

Gimp 3 is under development, but not available in the installation channels yet.

You can get a copy of the Gimp 3 development version as a flatpak from the website, then install GIMP-ML [github.io] from github and add the "addons" directory of wherever you installed GIMP-ML to the GImp3 preferences and this adds 16 AI functions to Gimp: things like semantic segmentation, inpainting, face recognition, and so on. You will also need to download the large model weights.

It takes about 4 GB of downloads for everything (depending on what you might already have), and getting everything to work is not completely trivial yet. Gimp3 is a moving target, and you will need to edit some of the python files and maybe move some things around, but any technically capable person should be able to figure it out.

Once Gimp3 is released I expect this add-on will be very popular. AI will be everywhere.

One criticism I have is with the quality. The Adobe photoshop generative fill [youtube.com] demos are very good, but I haven't been able to reproduce that quality with a local copy of stable diffusion. I suspect that the open source model weights are poor quality, and that Adobe has pruned, tuned, and enhanced their version for higher quality.

Generating a LLM is expensive. I saw one estimate of $75 million to generate a large model, and then once you have the model anyone can "tune" it with specific data (LORA tuning) to make "in the style of" whatever you tuned it with. Once you have the large model, tuning is cheap: a laptop with a beefy video chip can do this in a weekend.

The LLAMA model (the LLM made by Meta/Facebook) leaked online in the beginning of March, and in the intervening 2 months the open source crowd made tremendous improvements in the technology; imagine something like 5 years of improvements [semianalysis.com] in 3 months. I think LLAMA was a language model and not a RNN for images, which is why locally run stable diffusion isn't as good as the professional versions.

I strongly suspect that by the end of this year some very high quality large AI models will be available to everyone: leaked or released.

Everyone and their dog will be making AI creations of image, video, and audio.

(And note with only a little bit of cynicism: just in time for the 2024 presidential election.)

I love the fact...

[Rei]

... that anyone can do this. Maybe not with GPT-4 quality, but with the right tools and learning curve anyone can make their own models. I'm currently making a very lightweight (7B) summarizer model that takes long chunks of text and condenses it down to a specified target length (the goal is to be able to condense long contexts into their key points which will can fit into the token limits of lightweight, user-runnable models).

I get a giggle at watching long vacuuous screeds get condensed down into just a sentence or two. For example:

Well, I'm a few days removed from the St. Alban's EYC mission trip to Tennessee, and I still can't effectively communicate what exactly the trip meant. Certain words continue popping up--such as "incredible" and "amazing"--but I'm unable to put intellible sentences together! Any detailed account of what happened moment-by-moment during the trip will be both overwhelming and a bit too much to read and write in one sitting. Therefore, I'm going to avoid writing of the daily happenings. (If you're interested in that, please feel free to contact me though!) Instead, over the next few entries, I'm going to share certain happenings and lessons that I took away from the experience . With that, let me open by saying that I sure hope this past week wasn't simply an experience . It pains and worries me to think that the youth and sponsors will walk away from this trip thinking to themselves, "Boy, that was such a great experience!", or "I'm glad I experienced that!" An experience comes and goes and not much is to be said of it in the long run. A tetanus shot--now that's an experience! You get one and its either not so bad, or it kicks your butt! And you walk away saying, "I hope I don't have to experience that for a good long while!" Experiences, whether they are good or bad, hit us big at first but become distant memories later on. So to think that we'll walk away from the shadows of the Appalachian Mountains and say, "What an experience! I'm sure glad I got to see that!"...well, it scares me. Its my prayer that each member of the team met Jesus that week...and will be freshly and forever changed as a result. For some folks on the trip, it may have been the 100th time to witness Christ....but for others, it was their very first time. And no matter how many times it happens, each time someone runs into Jesus, history is forever changed. Let's look at some examples: 1. The Apostle Paul, who is accredited with leading and starting the first churches and writing most of the New Testament, was once a persecutor of Christians. He was even at the stoning of Stephen! (Acts 7:54-8:1) But something happened shortly after he took part in the murder of Stephen. Paul met Jesus. (Acts 9) Its quite incredible...for in the very passage that Paul was met by Christ, it opens with Meanwhile, Paul was still breathing out murderous threats against the Lord's disciples. Two verses later, the Lord knocked Paul right off his feet and onto the ground. Paul was speechless and blind! It goes on to say in verse 20 that after he regained his strength, he immediately set out to the Lord's work! Thanks be to God! 2. In John 4 , the Samaritan Woman encounters Christ. As she went to the town well to draw water, she came upon Christ sitting at the well. This must have been a very intimidating experience b/c not only was she a woman...but a Samaritan woman in the midst of a Jewish man. Furthermore, she had basically shared her bed with several men...and not even as their wife! But Christ meets her where she is nonetheless, and in verse 28 it tells us she drops the water jug where she is and goes back to share the Good News she has heard. It doesnt' stop there either! Just read verse 39!! It tells us that many more Samaritans came to know Christ as a result of that once lowly, dirty woman! Paul and the woman at the well were living shameful lives...and were headed on a collision course with disaster. Paul was responsible for the deaths of many Chr

Re:

[Rei]

To be clear, I haven't started the training yet, I'm just generating the training dataset... doing that with a 30B model, because you always want your training source to be a better model than what you're training. There's three parts to the training dataset:

* Good examples of accurate condensation to a specified length
* Accurate summaries, but wrong lengths
* Right lengths, but inaccurate summaries / bad takes.

Since even the 30B model isn't capable of targeting a specified length, I