If I use a QUERY job to take data from a source (e.g. a SQL database) and output it to a different (non-Mitto database) source, such as Snowflake – does Zuar and/or Mitto and/or Mitto’s Postgres Database have the ability to “see” the data?
Asking as we are trying to decipher what third parties could potentially access our datasets during the ETL process (using Mitto as the ETL tool) while not using Mitto’s Postgres as the database.
Great questions! If data is not being output to Mitto’s PostgreSQL then the database itself will not be able to see the data. However, since Mitto has to ingest and output data with a query job, it will have to pass through Mitto itself. It will not necessarily store this data unless the job is configured to create a store but it will log metrics about the data in Mitto’s Database, which includes things like number of records being ingested, traceback logs, exit codes/error codes if there is a problem.