txt-sushi

txt-sushi is a collection of command line utilities for processing comma-separated and tab-delimited files (AKA flat files, spreadsheets). By focusing exclusively on processing text files with a tabular structure, txt-sushi simplifies common tasks like filtering, joining and transformation that would take some effort to accomplish with a more powerful scripting language.

License

The source code is available under the BSD open source license.

Documentation

txt-sushi is still under active development and has not yet reached 1.0. This means that you can expect it to be a little bit rough around the edges but it should generally work as advertised.

Building/Revision Control/Issue Tracking/Announcements:

tssql:

The tssql utility allows you to use a subset of the SQL SELECT grammar on your flat files. See the tssql documentation for details.

csvtotab:

Converts a comma-delimited file to a tab-delimited file. This command takes one argument which is either a '-' for standard input or a filename.

csvtopretty:

Pretty format a comma-delimited file. Formats columns so that all data in column lines up (see examples above). This command takes one argument which is either a '-' for standard input or a filename.

transposecsv:

Transpose a CSV file. This command takes one argument which is either a '-' for standard input or a filename.

tabtocsv:

Converts a tab-delimited file to a comma-delimited file. This command takes one argument which is either a '-' for standard input or a filename.

tabtopretty:

Pretty format a tab-delimited file. Formats columns so that all data in column lines up. This command takes one argument which is either a '-' for standard input or a filename.

transposetab:

Transpose a tab-delimited file. This command takes one argument which is either a '-' for standard input or a filename.

namecolumns:

Adds a header row to a CSV file. Uses column names like col1, col2, col3... This command takes one argument which is either a '-' for standard input or a filename.

Links

There are many other great tools that help you work with flat files. Email me (keithshep@gmail.com) if you think there are other tools that should be added to this list:

Utility Notes
csvfix Very similar features to txt-sushi, uses many small utilities rather than SQL syntax
MS Log Parser Does not join tables, only works with MS Windows, closed source, works with many more data types than just flat files (XML, windows registry ...)
Fsdb Similar features to txt-sushi but uses many utilities rather than SQL
shsql Intended to work on mutable persistent CSV data where txt-sushi is more inteded to work on "data streams".
HaExcel An interesting tool that can be used to intelligently refactor spreadsheets into real databases
MySQL CSV Storage You can now use CSV files as the storage backend for MySQL