IO Job - CSV output with double quotes

Runner
#1

Hi team,
How can I tell Mitto IO job to create CSV files with double quotes? (" ")
I am using the JSON below to export an analytics table into CSV, but the end result is coming with no quotes CSV. FYI, I am on Mitto v2.8.10

Best,
Shawn

{
    "input": {
        "dbo": "postgresql://localhost/analytics",
        "query": [
            "select * from rds.sample_table1;"
        ],
        "use": "query.io#QueryInput"
    },
    "output": {
        "delimiter": ",",
        "path": "/var/mitto/data/sample_table1.csv",
        "use": "call:mitto.iov2#tocsv"
    },
    "steps": [
        {
            "transforms": [
                {
                    "rename_columns": false,
                    "use": "mitto.iov2.transform#ExtraColumnsTransform",
                    "include_empty_columns": true
                }
            ],
            "use": "mitto.iov2.steps#Input"
        },
        {
            "transforms": [
                {
                    "use": "mitto.iov2.transform#FlattenTransform"
                }
            ],
            "use": "mitto.iov2.steps#Output"
        }
    ]
}
#2

Hi Shawn,

If you want your tocsv job to quote all fields and headers like so:

"id","name","role","column_4"
"2","justin","CPO",""
"3","dylan","software engineer",""
"1","andy","Developer",""

you would use an output like this in your config above:

    "output": {
        "delimiter": ",",
        "path": "/var/mitto/data/sample_table.csv",
        "use": "call:mitto.iov2#tocsv",
        "quotechar": "\"",
        "quoting": 1
    },

See the official python csv documentation for all available parameters. Mitto will pass these thru from the config to the csv writer on run time.
https://docs.python.org/3/library/csv.html#csv-fmt-params

For the quoting param: 1 quotes all fields and headers, 2 quotes all non-numeric fields, 3 quotes nothing, and 0 (the default) quotes only those fields which contain special characters such as the delimiter.

Hope this helps.

1 Like
#3

Thank you, Andy. It worked like a charm.

1 Like