[Webinar] Unlock the Power of Mitto's Automation

Thanks for joining our webinar, Unlock the Power of Mitto’s Automation! For the on-demand version: Webinar On-Demand

Here are some resources that were discussed during the session:

If you have further questions related to automation, job types, specific use cases, etc., don’t hesitate to ask/reply on this thread or start your own topic! I can almost guarantee that someone has or will have the same question. Your contributions will pave the way for a stronger community around the tool!

As always, thanks for you support in making Mitto the powerful tool that it is today!

Here are some of the jobs that were used during the webinar:

Download file using curl job

Title: [curl] download file - collections.csv
Type: curl
Job Conf:

{
    "args": [
        "-o",
        "collections.csv"
    ],
    "url": "https://raw.githubusercontent.com/zuarbase/data/master/collections.csv"
}

Load CSV file into database using io job

Title: [CSV] collections.csv
Type: io
Job Conf:

{
    "input": {
        "delimiter": ",",
        "encoding": "ASCII",
        "includes_header": true,
        "source": "/var/mitto/data/collections.csv",
        "use": "xsv.iov2#XsvInput2"
    },
    "output": {
        "dbo": "postgresql://localhost/analytics",
        "schema": "csv",
        "tablename": "collections",
        "use": "call:mitto.iov2.db#todb"
    },
    "steps": [
        {
            "transforms": [
                {
                    "use": "mitto.iov2.transform#ExtraColumnsTransform"
                },
                {
                    "use": "mitto.iov2.transform#ColumnsTransform"
                }
            ],
            "use": "mitto.iov2.steps#Input"
        },
        {
            "use": "mitto.iov2.steps#CreateTable"
        },
        {
            "transforms": [
                {
                    "use": "mitto.iov2.transform#FlattenTransform"
                }
            ],
            "use": "mitto.iov2.steps#Output"
        },
        {
            "use": "mitto.iov2.steps#CollectMeta"
        }
    ]
}

Model the data with a sql job

Title: [SQL] Drop/create table example
Type: sql
Job Conf:

{
    "dbo": "postgresql://localhost/analytics",
    "kwargs": {},
    "parameters": {},
    "split": false,
    "sql": [
        "/* Create the database schema if it doesn't exist */",
        "CREATE SCHEMA IF NOT EXISTS model",
        ";",
        "",
        "/* Drop the database table if it exists */",
        "DROP TABLE IF EXISTS model.collections_summary",
        ";",
        "",
        "/* Create the data model table */",
        "CREATE TABLE IF NOT EXISTS model.collections_summary AS",
        "SELECT",
        "    state",
        "  , DATE_TRUNC('month', invoice_date)::DATE AS invoice_month",
        "  , SUM(amount)                             AS amount",
        "FROM csv.collections",
        "GROUP BY",
        "    1",
        "  , 2",
        ";"
    ],
    "transaction": true
}

SQL:

/* Create the database schema if it doesn't exist */
CREATE SCHEMA IF NOT EXISTS model
;

/* Drop the database table if it exists */
DROP TABLE IF EXISTS model.collections_summary
;

/* Create the data model table */
CREATE TABLE IF NOT EXISTS model.collections_summary AS
SELECT
    state
  , DATE_TRUNC('month', invoice_date)::DATE AS invoice_month
  , SUM(amount)                             AS amount
FROM csv.collections
GROUP BY
    1
  , 2
;

Sleep 10 seconds using cmd job

Title: [cmd] Sleep 10
Type: cmd
Job Conf:

{
    "cmd": "sleep 10",
    "cmd_env": {},
    "exec": false,
    "shell": true
}

Note: This was job id 12 and is referenced by other jobs below.

Start a job with another job using curl in a cmd job

Title: [cmd] curl start Mitto job 12
Type: cmd
Job Conf:

{
    "cmd": "curl -X POST -H 'accept:applicaton/json' -H 'Content-Type: applicaton/json' -d '{\"action\": \"START\"}' 'http://localhost:7127/v2/jobs/12/:actions'",
    "cmd_env": {},
    "exec": false,
    "shell": true
}

Start a job with another job using the Mitto cli in a cmd job

Title: [cmd] Mitto cli start Mitto job 12
Type: cmd
Job Conf:

{
    "cmd": "mitto start 12",
    "cmd_env": {
        "LANG": "C.UTF-8"
    },
    "exec": false,
    "shell": true
}