Using a Mitto regex job to pull in data from several files matching a regular expression comes with a few challenges. Previous to v2.9 it was difficult to tell which files matched a pattern, and whether those files used the same encoding you set in the job wizard. Furthermore, because Mitto will skip a file if it hasn’t been modified since the last run, was the matching file processed or not?
To help troubleshoot regex jobs, we’ve added a bit of logging at the start of the job run to provide some of this information. As the job is running, or in the event you’re having trouble getting the data you expect, click on the “Logs” button and the new output will be at the top of the logs.
For the regex pattern .+andy.csv you might see the following:
[2021-04-05T23:03:36] INFO [flatfile.iov2.regex] Matching files:
[2021-04-05T23:03:36] INFO [flatfile.iov2.regex] 2020-april-andy.csv - UTF-8-SIG - Last Modifed: 2021-03-16 22:23:14.155865+00:00 - No new changes
[2021-04-05T23:03:36] INFO [flatfile.iov2.regex] 2020-august-andy.csv - UTF-8-SIG - Last Modifed: 2021-03-16 22:23:14.763892+00:00 - No new changes
[2021-04-05T23:03:36] INFO [flatfile.iov2.regex] 2020-january-andy.csv - UTF-8 - Last Modifed: 2021-03-17 16:13:13.588050+00:00
[2021-04-05T23:03:36] INFO [flatfile.iov2.regex] 2020-december-andy.csv - ASCII - Last Modifed: 2021-03-16 22:23:15.131908+00:00 - No new changes
[2021-04-05T23:03:36] INFO [flatfile.iov2.regex] 2020-february-andy.csv - UTF-8-SIG - Last Modifed: 2021-03-16 22:23:15.679933+00:00 - No new changes
[2021-04-05T23:03:36] INFO [flatfile.iov2.regex] -----