Actually, you can solve all of the problems that you mentioned with a CSV format. If there a commas in the field, then you add quotes around the field. If there are quotes in the field, then you transform each quote into a double quote. If there are newlines then you have a couple options. You can escape them -- transform a newline into backslash-n. Or, if your parser handles it, you can just add quotes around the field and keep the newline, assuming the parser is smart enough to handle fields that are on multiple lines.
To handle optional fields just make the first line in the CSV file, a list of the column names. So the first line defines what follows in the CSV file.
The problem with CSV files is if you need to have metadata in the file itself, or if the data does not easily fit into a single table format.
Not only is UNIX dead, it's starting to smell really bad. -- Rob Pike