Do your customers know more about your data than you do?

There are many reasons for data discrepancies in your business that may not be obvious to anyone but your customers.

• Human error in manual processes
• Hidden bugs in billing, payment, or return/refund processes
• Hidden bugs in core applications that don’t cause an error
• Forgotten partially completed tasks

The concern isn’t just the revenue you may be loosing, but your reputation and business image could be impacted, making referrals hard to get.

What can you do?

• Put a consistent, automated data feed in place from your data sources
• Feed the data into a single source for analysis
• Use a flexible data analysis tool to explore and look into your data
• Automate and schedule reports for maintenance once everything has been validated and corrected.
• The key is to stay focused on the initial problem or analysis and take notes to follow up on the many other things you will find along the way.
• Get the initial data analysis for the most urgent problem started in a few days.

What would that cost?

A very robust and low risk system for a medium sized company should be around $10K or less per month. This can be up and running for initial analysis within a month and iterated on through time to build out a complete solution. Once the initial validation of processes and correction of any problems is complete, the data process will be in place to monitor and analyze for strategic growth.

What are the exact steps?

• Check to see if your data sources are available as a standard connection at https://fivetran.com/directory (if they are not then you will need to do more than this quick article presents)
• If they are then sign up for a trial account for your data storage at https://snowflake.com
• Create a Fivetran trial right in Snowflake https://fivetran.com/blog/how-to-set-up-fivetran-through-snowflake-partner-connect
• Choose and set up your connectors in Fivetran and start syncing data
• Get a demo and trial account with Looker at https://looker.com/
• At this point you will need someone with a minimum of good SQL skills in order to configure Looker using their lookml modeling language. It is definitely worth the effort to set up the models and explores that will generate SQL on the fly and allow flexible exploring of your data as well as dashboards, scheduled reports and all the other normal reporting stuff.
• Create views, models and explores in looker.

Technical qualities to look for in hiring an Analytics Engineer and a couple articles:

• 3-5 years of experience as a SQL Developer or similar role
• Excellent understanding of SQL programming
• Understanding of set based thinking
• nice to have, lookml experience
• https://blog.getdbt.com/hiring-analytics-engineer/
• https://blog.getdbt.com/what-is-an-analytics-engineer

Here are a few links for learning Looker and lookml:

• https://training.looker.com/
• https://looker.com/guide/getting-started
• https://docs.looker.com/data-modeling/learning-lookml

Extra Credit:

• For next steps, look into dbt cloud to move transforms that are well defined in Looker to dbt to create a more robust and performant solution. dbt will also add some additional functionality such as the ability to incrementally process data marts, run tests and a bit more code reuse than lookml provides.