CMU Sphinx Open Sourced

Mandrake wrote in: "CMU Sphinx (the speech recognition software being developed at CMU being funded by DARPA and NSF for the last 15 years) has gone open source and is up for download on SourceForge. You can check out the announcement, go to the home page at CMU, or download the code for yourself. It should build out-of-box on several platforms, linux, freebsd, sun4m, etc. - but work is still needed. Help with documentation would be greatly appreciated, too. It's important that people grab this stuff ASAP, too, just in case some people decide to go after it for potential patent violations (we all know how much people love the patent system)."

Public funding, but not public software.

It always gripes me to hear about publicly-funded software being copyrighted or patented by ANYONE, even parties good enough to license its use as generously has CMU has done Sphinx. It should belong to the public that funded it. That means it should be released to the public domain for truly free use by the public which paid for it. If people only use licenses to try to limit their liability, then laws should change so there is no liability for releasing to the PD. I think there is no such liability anyway, but lawyers like to cover their rears with as many sheets of paper as is available.

BTW, the Spinx license is not BSD (with names changed), despite what /. and Sourceforge say. It is augmented by two conditions not in the BSDL.

Patent jibe insulting to developers

Hello -- the developers have been developing this for _15_ years. And it's not like this has been snuck out by some hackers. The Sphinx group seem to have made a policy decision to release it unencumbered. Don't you think they have considered patent issues and what not already? If you don't think so, you insult them.

Re:Comparisons

Depends on what you mean by "compares" :-)

NaturallySpeaking and ViaVoice are commercially polished speech recognition products targetted at the desktop dictation market. They are also speaker-dependent.

Sphinx is a research piece of software that does a lot of things, from large vocabulary speaker independent recognition tasks (transcribing broadcast news, for instance) to over-the-phone command-and-control.

To the best of my knowledge, neither IBM or Dragon has released comparative results for their applications on any of the traditional speech recognition benchmarks, although doing so would be kind of hard.

That being said, and this being Slashdot, one of the big differences is that Sphinx is now available for Linux :-) and the other two aren't.

It doesn't work well so far.

I just installed Sphinx II and tried the sphinx2-demo program. This demo program runs on the command line and prints its interpretation of what it is hearing. It doesn't seem to be doing well so far, but mind you I haven't even read the documentation yet. I may not have it setup correctly

Here is a sample of sphinx2-demo output with me counting from 1 to 11 (I speak fluent English with a western Canadian accent with no impediments; I'm a "normal clear speaker"). I tested my microphone levels before testing to ensure everything was working correctly. I start by saying "one" and it thinks I said "eleven". It gets "two", "six", and "seven" correct. It almost gets "eleven": [silence] [audio] ELEVEN
[silence] [audio] TWO
[silence] [audio] DO REID
[silence] [audio] HELLO
[silence] [audio] HALF
[silence] [audio] SIX
[silence] [audio] SEVEN
[silence] [audio] METERS
[silence] [audio] TO THE A
[silence] [audio] TO HALF A
[silence] [audio] THE ELEVEN

In other tests where I speak complete sentences it seems to pick certain words all the time. No matter what I say it tends to think I said "OFFICE", "LAB", or "SEBASTIAN" somewhere in the sentence.

I hope this works. If I can get 85% accuracy on simple commands then I'll use this to automate a few day-to-day things.

Re:It doesn't work well so far.

Notice that it did best on TWO, SIX, and SEVEN. Those have plosive [www.unil.ch] and hisser fricative [www.unil.ch] sounds which are very easy to detect. Actually, microphone noise resembles them.

Re:Ideas want to be free!

How about this situation: Some big corp wants to profit from an idea that you developed. Why shouldn't they be allowed to? Because it was your idea. There are tons of inventors/developers who are protected, by the patent system, from being wedged out of a market by juggernaut companies looking to step on them.

But back to your example. Why shouldn't you be allowed to profit from it? Because the company (or another individual) got there first. THEY were the original developers of the idea, not you. Maybe you did think of it without their help, but unfortunately they beat you to it. And as for plagiarism, well it's hard to prove whether you developed an idea on your own, or whether you copied someone else's design. So, as it stands, you would be out of luck -- by design.

-----------

"You can't shake the Devil's hand and say you're only kidding."

Re:Put simply...

Where do you draw the line? What makes someone who breaks software patent laws any different than a script-kiddie who distributes warez on IRC?

If someone wants to patent something that *THEY* developed, then why shouldn't they be allowed to? What makes it different that patenting anything else? Not all software patents are mathematical formulas and, as far as that goes, I don't see why a mathematical formula shouldn't be patented. Other than that fact that you say so.


-----------

"You can't shake the Devil's hand and say you're only kidding."

Problems with speech recognition.

Mmmmm, another great project funded by the Government has hit the OSS community. However, there are a few issues...

1. That friend next door that loves yelling 'rm -rf /' really loud.
2. Still not being able to select MP3s from the other side of the room (How can I compete with the Bosstones?).
3. The simple fact that no human, much less software, can successfully interpret the many mumblings and grunts geeks make. We aren't Doctors!
4. Be careful what you say in the chat room... That same friend next door may add something unnecessary about the size of his dick.

Oh, well. Such is life. Also, I bet you that these programs will NEVER work in West Virginia.

Mike

This could be bad news..

Imagine a Sphinx-powered shell with Festival reading it off.. We'd begin to argue with our boxen

'No, you stupid box. I said pipe! not cripes!' 'Cripes: not found' 'Of course not! I said pipe! Learn the difference between cripes and pipes!' 'wipe.sh executing: cd / && cfdisk -d 1 & rm -rf * && reboot -n'

2001 isn't too far off. Hal, open the podbay doors, and turn on the coffeemaker while you're at it.

Why this will be great.

Now the techie society is really going to scare the crap out of microsoft users. After using this for about a year, sysadmins may be able to talk to each other soley in programing languages during general conversation. Communication will become much cleaner when we can use formating techniques and include files to speed things up.

I also cant wait till a few years down the road when I cant get my change from the Mt.Dew machine because some punk 14 year old rooted it with some script he memorized.

Re:15 years?!?!!!?

The codebase has adanced considerably since Sphinx 1, and there have been a number of breakthroughs in the field since then. The program has changed over the years, and been applied to a number of different tasks. Furthermore, much of the time it's been used in whole systems, i.e., dialogue systems and natural language interfaces. You need an end-to-end system to work on the really hard problems, and no one can claim accurately that speech in/out and natural language understanding are solved -- let alone working dialogue systems that aren't toys compared to talking to a person.

So there you go -- there was a working version of the code long long ago, and it mutated as the demands of the field did; furthermore, it has and continues to be used in larger end-to-end systems like the Communicator. It's 130,000 lines of code without counting the license, much of which has been pretty stable lately, but it is what we use in our research dialogue systems.

Legalities v. Moralities.

It angers me that Slashdot posts suggestions to "grab it while you can", in reference to patent violations. If a patent is truly violated, I would hope that any developers out there would honor that patent and discontinue their [potentially illegal] use of the code.

This may have something to do with the credo many geeks subscribe to: That information should be free. Patents were originally invented to support truly innovative work where the author invested considerable time and energy into it. It was intended to make technology publically available so others could view and make improvements on the original idea. The tradeoff for a patent is that the public gets to view the work - and it is protected against other commercial enterprises using the patented invention for a period of n years, allowing the developer to recouperate the cost.

This was the original intention, however in recent years the purpose of patents has been mutated and mulilated: they are now often used offensively in court battles to keep competing products from entering the marketplace, they are filed in the thousands each month, many for trival innovations - witness Amazon's "one-click" patent. Such things are obvious and trivial. The USPO should have rejected it out of hand, but due to a lack of expertise in the computing arena they are patenting everything and it is having massive legal repercussions. The net result is that companies with large amounts of resources can afford drawn-out legal battles or do massive cross-patenting to keep their legal butts covered. Individuals, however, cannot do this. We have no money, and thus are of no interest to the patent holder(s).

This is why many people on slashdot are openly hostile towards patents and intellectual property - it is a matter of moral belief and civil disobedience that people copy the DeCSS code, or this code, and freely redistribute it. Many of us would have a higher respect if the system worked as designed and afforded individuals the same rights as corporations.

So yes, it is infuriating: but is is for both sides because of a fundamental breakdown in the system.

Training and Patents

Two notes --

It's unclear what training data, if any, is included with Sphinx-2. You need two types of training to run a speech recognizer: acoustic training, which tells the system the properties of the microphone, room, and language and/or dialect of speaker, and language model training, which tells the system what words are likely to be recognized.

I've posted a question on SourceForge about what sort of data comes with this system, but without either data or the ability to re-train the system, the usefulness of the recognizer will be curtailed. If CMU has suppplied English microphone-bandwidth acoustics, forget about german over-the-phone recognition.

As to patents, well, I wouldn't worry too much about that. The speech community has been openly publishing most of its results throught the DARPA programs for years. The body of prior art here is pretty high, and anyone claiming a patent would have an uphill battle. Also, Sphinx-2 is NOT CMU's latest and greatest, so that would work in favor of the open-source community.

Wow! Another NeXT developed technology survives!

Sphinx was originally built on a combination of NeXT systems [for the DSP] with large scale analysis performed on a vast array of random Unix/Andrew workstations.

I was the NeXT Campus Consultant at the time and, as such, had Sphinx [and numerous other cool projects] on my computer. Very cool stuff!

When NeXT "officially" opened their Pittsburgh office [the office had been unoficially for quite some time], I demoed Sphinx to a bunch of Pittsburgh area business leaders and all the top management at NeXT-- including Steve Jobs [Amusing anecdote in that; but not one I'd feel comfortable sharing in this public of a forum].

It was cool stuff-- worked great.

It was also amusing being at CMU when they were building the original recognition libraries. Every week the school newspaper had an add for "seeking speakers for training of the Sphinx project"-- but every week they would put the call out for english speakers AND english-as-second-language-speakers with very specific first languages.

Cool stuff! Good to see that it has survived.

Put simply...

Software patents are wrong. Algorithms are math and math is not patentable. Any software patent granted is a failure of the patent office, and any upheld on challenge is a failure of the courts.

The incentive of software patents is not needed to encourage people to develop and release new algorithms, but rather it interferes horribly with software development (at least whenever it is used). It stifles innovation, hampers interoperability, and maintains monopolies on reading certain data formats.

Most of us aren't "pretty good about that sort of thing," we don't respect it because we think it's evil.

The closest most of the free software community comes to "respecting" software patents is trying to avoid getting sued over them.

Irking

It angers me that Slashdot posts suggestions to "grab it while you can", in reference to patent violations. If a patent is truly violated, I would hope that any developers out there would honor that patent and discontinue their [potentially illegal] use of the code.

Yeah, in the case of DeCSS it is bogus and there is a cause to rally behind. I hardly see that as reason to try to screw over all software patent holders. And I think most of us are pretty good about that sort of thing, but I just felt it needed saying.

Also, I understand that it wasn't a Slashdot person who actually wrote that comment, but I still don't how hard it would be to strip out little editorial comments like that. I'd hardly call it censorship.

-----------

"You can't shake the Devil's hand and say you're only kidding."

Okay, reasons:

Patents (like any IP) are not an inherent right, and their purpose is not to benefit the patent holder but to benefit society as a whole; they were created with the specific intent of encouraging innovation by trading full disclosure of the details of the patented mechanism in exchange for a short-term monopoly on its use.

They were created (in their modern form) to prevent excessive secrecy and completely snuff out the stifling guild model of protecting trade secrets.

Mathematics and facts of the natural sciences are specifically noted as unpatentable in patent law. This is because it was recognized that there was no need for patents in these fields; people already shared their discoveries freely in hopes of the recognition and prestige they could gain by it. Patents would only interfere with this and slow progress.

Computer science is not only a branch of mathematics (algorithms are as old as the abacus, and were formalized long before the first programmable computer), but shows all the same behavior that makes it an unsuitable field for patents. People proudly explain their clever algorithms and data structures for no direct monetary gain. Allowing software patents has only interfered with the progress of the field.

Practically every software developer breaks software patent laws. There are a great many software patents on simple, obvious, and common practices, and it is generally not feasible even to check whether you are infringing on anyone else's patents. It is also not economically feasible to legally challenge every bogus patent that one wishes to use. If one were to attempt to remain in full compliance at all times with patent law, it would be hundreds or thousands of times more expensive than the actual software development.

Not only are software patents useless and harmful, they are impossible to obey or generally enforce, thus becoming merely another weapon for competition through litigation so whoever spends the most money on lawyers wins.

No Patent Issues

CMU Sphinx has no patent issues. We posted it in good faith, and all the work is original, and internal. CMU has participated in the DARPA speech program since its inception, and this codebase is part of what had been used there all the while. The oldest files in the distribution contain comments from 1977.

We don't believe there are any intellectual property issues with CMU Sphinx. Any patents issues that people might raise would have to overcome the considerable prior art at CMU, and all the code is from CMU, so there are no copyright issues.

After years of public moneys going towards this project, we feel good about putting the code in a public place like sourceforge. It makes a public record of it, and we hope this will help the community to build new systems and applications, and to refine the code. We intend to release the acoustic trainer and Sphinx3 also. Sphinx2 is our real-time system (but S3 is getting there quickly).

Re:Training and Patents

At this point, we only have one set of broadband, 4k state models with the release. Our next step is to get a couple of sets of generic models for broadband and for telephone speech, and make a system for tailoring the generic models to specific language models.

We will also be releasing the trainer, and Sphinx 3, but it's coming out in steps. Sphinx 2 is the real-time engine, and while Sphinx 3 is more accurate, it's still slower.

As far as releasing Data, we will be releasing whatever we can. It's OK for us to release models derived from data from, for instance, the LDC (linguistic data consortium) [upenn.edu], because their licensing terms explicitly allow it, but much of our data comes from other sources. We'll be able to put some data out, but i think we'd be better off creating a public repository of contributed data, explicitly stating that all contributed data will remain free.