Self-service analytics has been the holy grail of data teams for decades. We’ve all heard the promise: Empower business users to answer their own questions without constantly bothering the data engineering team. Yet despite years of investment in tools and training, most organizations are still stuck in what can only be described as the “Wild West” of data analysis.

So why hasn’t self-service analytics worked yet? And more importantly, how can we finally make it work?

The Current State: A Wild West of Data

If you’ve worked in data for any length of time, this scenario probably sounds familiar: An analyst gets a urgent request from their boss. They need answers fast, but the data engineering team has a six-month queue. So what happens? The analyst does whatever they can to get to their answers.

They jump between tools, copy logic from old queries, create custom calculations in Excel and somehow piece together a report. Meanwhile, their colleague down the hall is doing the exact same thing — but with different definitions, different data sources and different assumptions.

The result? A complete breakdown of trust in data.

This isn’t anyone’s fault. It’s a systemic problem created by misaligned incentives and organizational silos. Data engineers are incentivized to build core assets for the entire company in a well-governed way. Analytics teams are incentivized to get their job done as quickly as possible to serve their business stakeholders.

These conflicting priorities create a perfect storm: Analysts can’t wait for engineering queues, so they build their own solutions. Engineers can’t accommodate every ad-hoc request, so they focus on infrastructure. The two teams end up talking at each other instead of with each other.

The Universal Tension: Governance vs. Empowerment

At the heart of the self-service analytics problem lies a fundamental tension that every organization faces: The pendulum between governance and empowerment.

On one side, you have the need for governed, trusted, consistent data assets. On the other side, you have business users who need the flexibility to slice and dice data in ways that serve their specific needs. Traditional approaches have forced organizations to choose one or the other, inevitably leaving someone frustrated.

The engineering team says, “We need six months to build this properly with all the right governance and quality checks.”

The analytics team says, “I need this report by Thursday, and I’ll figure out my own way to get there.”

Neither side is wrong, they’re just operating under different constraints and priorities.

The Missing Pieces: Why Previous Attempts Failed

Looking at failed self-service initiatives, several common patterns emerge:

Lack of unified language: Different teams use different tools, different terminologies and different approaches to the same problems. When an analyst says they need “customer data,” and an engineer hears “customer data,” they might be talking about completely different things.

Trust deficit: Without visibility into where data comes from, how it’s calculated and who’s responsible for it, business users lose confidence. If you can’t trust the data, you can’t have true self-service.

Tool fragmentation: Traditional workflows require users to jump between multiple tools: one for data discovery, another for analysis, a third for transformation and a fourth for visualization. Each tool switch introduces friction and potential errors.

Technical barriers: Many self-service tools either require deep SQL knowledge (excluding business users) or operate in complete isolation from engineering workflows (excluding governance).

A New Paradigm: Collaboration Without Chaos

The breakthrough insight is that self-service analytics doesn’t mean analytics teams working in isolation. It means analytics teams and engineering teams working together under a shared framework.

This is exactly what dbt has been building toward with their latest platform developments. Their approach recognizes that the solution isn’t choosing between governance and empowerment, but creating infrastructure that enables both simultaneously.

Imagine a world where:

• Data engineers continue to build high-quality, well-governed core data assets that serve the entire organization.
• Analytics teams can easily discover and reference those core assets, then extend them for their specific business needs.
• Everything happens within the same framework, so engineering teams have full visibility into how their data is being used downstream.
• All transformations resolve to SQL, meaning they can be version controlled, audited and governed just like any other code.

This isn’t about replacing data engineers or making analysts learn complex programming. It’s about creating a collaboration model where both teams can work at their optimal skill level while maintaining the governance and quality standards the organization needs.

Three Pillars of Modern Self-Service

1. Universal Data Discovery – The journey often starts with a simple question: “What data should I use to answer this business question?” Modern self-service requires a global catalog that spans all data assets — not just dbt models, but everything in your data warehouse.

dbt’s new Catalog experience exemplifies this approach. It provides global search across all dbt projects while also scanning and indexing data warehouse assets from platforms like Snowflake. Users can search, understand and evaluate data assets with full lineage, ownership and quality indicators before committing to use them. When you can see that a table has passing tests, clear documentation and active ownership, trust starts to rebuild.

2. Iterative Analysis and Validation – Analysts are naturally skeptical. They don’t trust data until they can get their hands on it and see what’s inside. The traditional approach required switching between tools: one for discovery, another for analysis, a third for visualization.

dbt Insights addresses this by embedding analysis capabilities directly into the discovery workflow. Users can run quick queries against data assets, create visualizations, and validate their assumptions without leaving the platform. Even better, AI assistance through Copilot allows analysts to use natural language to generate SQL queries, making the analysis accessible to users regardless of their technical background.

3. Governed Asset Creation – When analysis reveals the need for a new data asset, that asset should be created within the same governance framework that engineering teams use. This is where dbt Canvas shines by providing a drag-and-drop, visual interface for building data transformations that still resolve to SQL under the hood.

The key insight here is that Canvas isn’t a separate tool that creates ungoverned assets. Everything built in Canvas goes through the same code review, testing, and approval processes that engineering teams use. A business analyst can use visual interfaces to build a model, but it still gets version controlled, tested and documented just like engineer-written SQL.

The Power of Shared Language

One of the most important breakthroughs is creating a truly shared language between technical and business teams. This isn’t just about everyone learning SQL (though that helps). It’s about creating a common framework where:

• Business users can express their needs in terms that engineers understand.
• Engineers can build assets that business users can easily discover and extend.
• Everyone can see the lineage and dependencies of data transformations.
• Documentation and governance happen automatically as part of the workflow.

dbt’s approach to this shared language is particularly compelling. By having everything resolve to SQL — whether it was built by a data engineer writing code, an analyst using Canvas’s drag-and-drop interface or someone using AI assistance — all stakeholders can understand what’s happening under the hood. The visual tools lower the barrier to entry, but the SQL foundation ensures transparency and governance.

When teams speak the same language, they can finally move from working in parallel to working together.

Looking Forward: Empowerment Without Compromise

The future of self-service analytics isn’t about choosing between empowerment and governance: It’s about achieving both simultaneously.

What we’re seeing with modern platforms like dbt is a fundamental shift in how this problem is approached. Instead of building separate tools for different user types, the focus is on creating unified platforms that meet users where they are while maintaining organizational standards.

A business analyst can use dbt Canvas’s visual interface to build a customer cohort analysis by dragging and dropping transformations. A technical analyst can write complex SQL for the same project. Both approaches create assets that are version controlled, documented and governed identically. More importantly, both users can collaborate on the same project with full visibility into each other’s work.

This represents the end of the false choice between accessibility and governance. When an analyst creates a transformation using Canvas, it automatically generates documentation, enables lineage tracking and can be reviewed by engineering teams using the same processes they’d use for any other code change.

The End of the Wild West

Self-service analytics has failed for decades because we’ve been trying to solve the wrong problem. We’ve focused on building tools for individual users rather than frameworks for collaboration. We’ve emphasized either empowerment or governance rather than finding ways to achieve both.

The organizations that succeed with self-service analytics in the coming years will be those that recognize this fundamental shift: From isolated self-service to collaborative self-service, from tool proliferation to platform consolidation, and from choosing between governance and empowerment to achieving both through better architecture.

Platforms like dbt are leading this transformation by creating environments where data engineers can maintain their focus on core infrastructure while analytics teams gain the ability to extend and build upon that foundation. The result is faster time-to-insight for business teams without sacrificing the data quality and governance that organizations need.

The Wild West era of data analysis doesn’t have to be permanent. With the right approach, and the right tools, we can finally deliver on the promise of self-service analytics: Empowering business users to get the answers they need while maintaining the trust and governance that organizations require.

To see how dbt is helping the push toward true modern self-service analytics, check out this demo from a recent InterWorks/dbt webinar (available here):

We also have a German language version of this webinar as well! If you want to talk shop about the future of self-service data analytics, or maybe you want to check out some of dbt’s offerings, drop us a line and see what we can do for you.

The post Modernize Self-Service Analytics with dbt appeared first on InterWorks.

dbt is continuing its tradition of upping the ante and developing new features to really enhance the seamless data transformation experience. In their most recent installment of “What’s new in dbt Cloud,” there were many announcements. Here are a few that we are the most excited about!

SDF Acquisition

To start things off, dbt acquired SDF Labs. To directly quote dbt, it will improve the developer experience through “100x faster performance, the ability to validate code in dev to boost data quality and optimize compute costs, and rich metadata to improve lineage and enable nuanced governance use cases.” That sounds like a good deal to me!

Semantic Layer Integrations

dbt’s semantic layer has been available for integration for a few different BI tools, Tableau most notably. Sigma can now be added to that list! The preview of this is out and ready to be tested. This will allow users to connect to the metrics set in the semantic layer in dbt, creating a governed and trusted data visualization.

You can now bring in “query_with_all_group_bys” to return all valid dimensions for a given set of metrics. This will be helpful in removing the guesswork and adding some transparency for those using the semantic layer.

Auto-Exposures

This particular function is in beta, but dbt can now be used to automatically orchestrate the refreshing of Tableau dashboards when new data is available. This, to me, is one of the things I’ve been looking forward to since auto-exposures was announced.

Platform

“Release tracks” (which is a slight changing and renaming of the “versionless” options for dbt instances) allows you to manage dbt version upgrades. The default is “Latest,” which is what it is now. However, they are also adding two other options called “Compatible” and “Extended” which will let there be some customization/delay in upgrades as the customer needs.

Dark mode is now available in Preview. Need I say more?

New account homepage will allow you to “favorite” your projects to get to them quickly, as well as showing some general information about the projects like the number of models, number of sources, and documentation coverage.

In Conclusion

These are just a few of the newest additions to dbt Cloud. Come back in April when I will write another summary after their dbt Cloud Launch Showcase! And if you are interested in working with us on your dbt needs, feel free to reach out and see what we can do for you!

The post What’s New in dbt? appeared first on InterWorks.

dbt presented their “dbt Cloud Launch Showcase” last week. In it, the team rolled out a buffet of new features and upgrades designed to make data transformation smoother than ever. Whether you’re knee-deep in SQL or just trying to keep your data ducks in a row, these updates promise to make your life a whole lot easier. dbt is clearly on a mission to keep raising the bar and ensure that data nerds (like myself) everywhere have the best tools at their fingertips. This is not an exhaustive list by any stretch of the imagination, but these are the things I am the most excited for — and think you will be too.

dbt Assist

The ability to use AI to generate documentation and tests has been introduced to the development process. This is currently in beta and the demo of it was really impressive! Once you create the model SQL, you can use the “dbt Assist” button to generate the ymls for the model and tests. It is even able to identify the primary keys and apply the correct tests to them.

Advanced CI Functions within Pull Requests

A dbt Cloud CI report will now be created that has some CI checks as well as a “compare changes” option. This will show you previews of how your data changes after your code changes:

 Automatic Exposures

Exposures were released as part of dbt a while ago but now they’re introducing automatic exposures (coming mid-July for Tableau and then PowerBI afterwards). Once set up, you’ll be able to do some helpful things such as:

• See what workbooks/reports are utilizing the model.
• Sync it so that the Tableau data is automatically updated once a model stops running (instead of scheduling the Tableau extract to hopefully update at the correct time).
• See impact of model changes on downstream dashboards.

dbt Explorer

This is exactly what it sounds like: a shared canvas for data developers and data analysts to collaborate. It allows users to look at column-level lineage, model performance analysis and be able to search through all the models in a visually friendly format:

Visual Editor

This was dbt’s version of Steve Jobs’ classic, “Oh, and one more thing,” at the end of Apple presentations. Starting “soon,” dbt will have a visual editor available. Users will be able to drag and drop connections and user friendly wrappers to perform transformations. There is even some AI assist as a part of it. If you don’t know how to write the formula you want, you can type a question/request and it’ll generate code. All of this will then be translated into the SQL model.

There is also the ability to take a model that was originally written in SQL and edit it using the visual editor. This is genuinely exciting and makes dbt even more approachable for non-technical users, allowing even more collaboration within and between teams.

If you’d like us to work with you on all your dbt needs, from getting started to fine tuning, feel free to reach out and see what we can do for you!

The post Highlights From the Spring 2024 dbt Cloud Launch Showcase appeared first on InterWorks.

Recently, Dataiku announced their 2022 class of Neurons—the highest performing and most passionate members of the Dataiku community. Beginning in 2020, those nominated to be Neurons are the movers and shakers in the Dataiku sphere, providing new users direction and guidance in their Dataiku journey.

A Neuron at InterWorks

We’re beyond pleased to share that InterWorks’ very own Rachel Kurtz was inducted into this class of Neurons along with 35 other experts from around the world! We recently caught up with Rachel to find out how she reacted to finding out that she was Dataiku’s newest Neuron:

Rachel: “I was excited. The program started just over a year ago and there were about 10 people there, so it’s still a fairly small group at this point. I’m very honored to be nominated for it, especially knowing that Dataiku themselves were the ones that nominated me … It was very humbling.”

Rachel also remarked that the new Neuron class is planning on attending conferences and get-togethers as a group, as well as strengthening their bonds through close collaborations on Dataiku projects.

Rachel: “I love Dataiku as a tool and using Dataiku, but above all, I really love helping people to also find that love of Dataiku and being able to use it. Just being able to have this title that labels me as somebody that you can come to if you have questions about Dataiku or need assistance to help promote it within your company, it makes me really happy that I can help others to fall in love with it, too.”

Rachel is the first InterWorker to be nominated for the new award. Aaron Confer, InterWorks’ Global Director of Business Development, and James Wright, InterWorks’ Global BI Practice Director, released a joint statement about Rachel and her new title:

“We’re delighted to celebrate Rachel’s induction into Dataiku’s select Neuron group!

Rachel is an amazing colleague and is certainly deserving of the honor. She’s been instrumental in building out our global capacity for data science, machine learning and self-service analytics. We’re excited to see where her leadership will continue to take our company, as well as our customers.

Overall, we see a bright future for Dataiku’s technology and our thriving partnership.”

For a full list of the 35 new Dataiku Neurons and an introductory video on the impressive roster of Neurons, check out Dataiku’s LinkedIn post or visit their blog to keep up with the newest happenings within the Dataiku community.

The post Meet InterWorks’ First Neuron: Rachel Kurtz appeared first on InterWorks.

“The goal is to turn data into information and information into insight.”

This quote from former Hewlett Packard CEO Carly Fiorina really sums up what we at InterWorks feel is everyone’s ultimate goal, regardless of whether they’re doing statistical analysis, data modeling, visualizations or data reporting. We’re all trying to gain insight into what that data is trying to tell us. Raw data inherently doesn’t have that information; you have to work with it, shape it, clean it and create calculations to truly glean those actionable insights you crave.

However, as we’ve spoken with many clients over the years, we’ve discovered that the final step of data prep is where a lot of obstacles emerge. Perhaps you’re currently experiencing some of these speed bumps as well!

Possible Challenges in Data Prep

A few examples of the issues our clients have faced are:

• They have data in a lot of disparate locations, whether they’re different warehouses, a combination of warehouses and local files, or even multiple tables in the same warehouse.
  • This has led users to workarounds such as custom SQL, multiple joins and/or blends in Tableau or slow, unsustainable Tableau workbooks due to an excessive number of calculations and LODs.
• They have employees and team members of various skill levels (some Python and SQL wizards, some who are not there yet) who are having trouble bridging the gap in order to work together, which contributes to duplicated work and miscommunication.
• Getting the data ready for investigation is a slow and often manual process with lots of download-to-Excel-and-edit.
• They’ve been hearing about data science, machine learning and AI and are very interested in using it but don’t know where to start.

If any of the previous problems resonate with you, may I please introduce you to Dataiku?

What Is Dataiku?

In Dataiku’s own words, it is one central, end-to-end solution for the design, deployment and management of analytics, ML and AI applications. Dataiku is infrastructure agnostic, working with all flavors of cloud and on-prem storage and compute. It is also inclusive of all skillsets, whether on the technical side working in code or on the business side with low to no code.

This is a very apt description of Dataiku, but I want to break it down even further and focus on two specific use cases: last-mile (or analytic) data prep and data science.

Last-Mile (or Analytic) Data Prep

When we discuss this type of data prep, it is not to be confused with enterprise-level ETL/ELT, which is often handled by Fivetran or Matillion. ETL/ELT is often utilized by Data Engineering/IT teams. Instead, last-mile/analytic data prep refers to the data preparation that happens before a particular report, visualization or analysis is created. This is more customized and often utilized by individuals or groups of data scientists, business analysts and data analysts.

Dataiku includes over 90 built-in data transformers for common data manipulations like binning, concatenation, currency conversions, date conversions, filtering, splitting, geospatial and more. Even when a transformer doesn’t exist in the library, users can quickly write formulas similar to those used in spreadsheets to accomplish almost any data transformation task.

Data Science

Dataiku’s original name was Dataiku DSS where DSS stood for Data Science Studio. It was created with the intention of having a central location accessible and usable by the entire range of data scientists, from those who are just starting their data science journey to those who have been on the journey for a while and write their own models in R or Python. If someone needs assistance in model creation, there is a Lab section of a workflow that walks the user through the steps and how to create it with the user-friendly UI. If someone prefers to write their own model, they can upload that instead.

Where Does Dataiku Fit?

To summarize the previous section, Dataiku is a low-barrier-of-entry tool that bridges the gap between data sources/warehouses and data visualizations/reports, allowing all levels of users to be able to enhance the data through last-mile data prep and/or model building. It empowers analysts to work with and build the data they need for their analytics, and it removes some of the burden on the data engineers:

Why Should You Care About Dataiku?

There are three key aspects of Dataiku to keep in mind when deciding if it is right for your company. These qualities will answer many of the problems you may be facing (enumerated above) and some you didn’t even realize you had:

1. Collaborative
   Dataiku was built with collaboration in mind. Through the Git integration inherent within Dataiku, multiple people can be working on the same project without having to worry about actively pushing or pulling through Git. There are also many internal documentation capabilities including, but not limited to, wiki pages, a discussion forum and a shared to-do list where you can tag coworkers.
2. Accessible
   As stated previously, Dataiku was created to be easily used by coders and non-coders alike. This keeps people and teams from being isolated in silos and allows cross-experience collaboration.
3. Cloud-based
   The cloud-based ­­­nature of Dataiku means that it can connect to many different data sources and warehouses in an efficient manner. It can also push the computation of every step of the process onto a database, making it so that you are not limited by or reliant on your local machine’s capabilities. A final ­­­advantage of Dataiku being cloud-based is that you can run any of the workflows you’ve created on a schedule, and you don’t even have to be logged in to your instance to do it.

How InterWorks Can Help!

With experts deeply familiar with Dataiku, we’re ready to partner with you and help guide your Dataiku journey, wherever it may lead you. Here are some specifics of how we can do that:

1. Implementation
   We can assist in the installation and implementation of Dataiku at your company.
2. Trainings
   We were the first of Dataiku’s partners to lead a training in their stead, and we continue to lead trainings to this day, both on site and virtually. Whether your company is brand new to Dataiku or needs more advanced training, we can tailor the right training plan for you.
3. Consulting
   With many consultants certified in Dataiku, we are ready to assist you in such tasks as building your first model, moving your manual last-mile data prep processes over or creating a Center of Excellence to help with adoption of Dataiku at your company.

Interested in seeing a demo of Dataiku or discussing it further with us? Reach out! We’d love to help you get started. You can also start a 14-day free trial here on your own.

Contact Us

The post Meet Dataiku, the Everyday AI Powerhouse for Your Data appeared first on InterWorks.

“I have a lot of data and want my users to be able filter the dashboard. However, there are a few options in the filter that I don’t want them to be able to select. How can I remove these from the dropdown but not filter them out when ‘All’ has been selected?”

This Tableau solution came about from the above client question, but there could be a few use cases when you might want to omit your filters from a dropdown like this. Perhaps you want to include a region in the data, but for security reasons, you don’t want users to be able to filter to that region specifically, when viewing HR data, for example. Maybe you want to see overall information about all the groups, but if a group has less than five employees, you don’t want the user to be able to filter that in order to protect anonymity. Another example could be that when looking at sales data, filtering to a specific region does not give a lot of helpful information, so it’s unnecessary for users to have that option.

When it comes to implementing a solution, there are a few options for this!

The first is to create a parameter and use that to filter the data. This is a great option. However, if the data changes (i.e. you get a new region in your data) you have to edit the parameter to include that. Also, parameters (as of 2021.3) do not allow for multi-select. So, we need another way to go about doing this.

The second way is to create an extra sheet for the dashboard we’re going to use to remove those options. It can then be added as either a title or hidden within the dashboard.

Create a Copy of the Field to Remove Options From

In our example, that is the Region field. I want my users to be able to either see the data from all regions or select South, East and/or Central. I do not want them to be able to filter down to the West region:

To do this, duplicate the field. You can leave the default name (‘Region (copy)’), but I prefer to rename and add to remove options from filter, it so I remember what the field is intended to do.

Add Original Field to Filters Card

Bring the original field (in our example, Region) to the Filter card on your sheet and select All:

Add Duplicated Field to Filters Card

Bring the duplicated field (Region – to remove options from filter) to the Filter card. Here is where you’re now going to change the filter type to Exclude and select the options you do not want your user to be able to select.

Important note: Excluding options, rather than selecting a static list, will give you the flexibility of new options automatically being added to the filter. If you want to keep a static list, regardless of new data, you should select only the options you want. For this example, we want to remove the option for the user to be able to select the West region:

Show Original Field Filter

Right-click on the original field in the Filters card and select Show Filter. This will allow us to select whether we want to see it as a drop-down, single-select, multi-select, etc.:

Show Only Relevant Values for That Filter

Now is when we start to see the exclusion come into play. Click on the drop-down arrow in the top-right of your filter, and select Only Relevant Values. This will remove the West region option because of the Region – to remove options from filter filter that we added:

Add Title to the Text in This Sheet

In this example, I’m going to use this sheet I’ve created to be the title on my dashboard. I do that by adding an ad hoc field to the Text box in the Marks card and then editing the Text:

Add Sheet to Your Dashboard

To now utilize this new filter, you need to bring this sheet to your dashboard. I already have a chart in my dashboard showing my Sales and Profits broken up by region. I’m going to put the newly created title to be above the chart. I’m also going to do a few formatting things, such as hiding the title of the sheet, make it fit the entire view and shorten it a bit:

Doing this brings in that new region filter that does not have West as an option.

Apply Filter to All Sheets in Dashboard

As of now, the region filter that was brought in only applies to that Title sheet. I would like it to apply to all sheets in the dashboard. To do this, click on filter and then the down arrow in the right-hand corner and select Apply to Worksheets > All Using This Data Source. You can also select specific worksheets it applies to if you don’t want it to apply to all:

You’re Done!

That’s it. You can play around with it and see that when you select (All), you will see the data from the West region, but you are not able to filter to it specifically:

Download the attached workbook if you want to dive into it further.

Note: There is a small bug in Tableau where if you deselect All then choose all three options individually, Tableau will check the All button, even though it’s only showing those three regions. If you clear it then click All, it will show up.

The post Hiding a Tableau Filter’s Options While Including Its Data appeared first on InterWorks.

Recently, we hosted a webinar that focused on the anchoring question of, What is data prep and what tools can help me with it?

“The goal is to turn data into information and information into insight” – Carly Fiorina, CEO of Hewlett Packard

We feel like this is everyone’s goal—anyone who is doing statistical analysis, data modeling, visualizations or reporting on data. You’re trying to gain insight into what that data is trying to tell you. Raw data inherently doesn’t have that information; you have to work with it, shape it, clean it and create calculations to really gain the insight from it.

Data Prep as the Key to Insights

The focus in this blog is how we move our data through to information and then to insight. This is where data prep comes into play. Here are some of the typical challenges/scenarios we see people facing:

• Data Engineering/IT has given you your data, but it’s not quite right for what you’re hoping to do.
• You don’t have a Data Engineering/IT team.
• Data preparation is a very manual process for you.
• Your Tableau connections are full of joins, blends or custom SQL.
• The number of LODs within Tableau is bogging down your workbooks or becoming unsustainable.

If any of these problems/scenarios speak to you, we recommend that you keep reading. We’ll look at how you can make your data prep efficient, repeatable and actionable, and we’ll discuss the tools that can help you do that.

Let’s review this typical data workflow with three unique data sources. We’ll be looking at the three tools we see as being at the forefront of the data prep landscape: Tableau Prep, Dataiku and Alteryx. Let’s look at the things you should be thinking about when you compare the three of them.

Understanding the Difference Between ETL and Data Prep

Before we get deep into the review, let’s discuss the difference between ETL (extract, transform and load) and data prep. We hear these two terms intertwined and although they do have similarities, we want to make the delineation between the two of them.

ETL (Extract, Transform, Load)

This process is extracting the data from multiple sources (typically different types of data) and transforming it, so this is shaping the data, creating calculations and finally loading it into an often-large data warehouse. This tool is often utilized by Data Engineering/IT teams and completed at a company-wide level.

Data Prep

More recently reframed as the “last mile data prep”, this is the prep before the visualization. Typically, this tends to be more customized and is often utilized by individuals or groups of data scientists, business analysts and data analysts.

So, when deciding between the three solutions above, it’s important to assess your answers to these questions:

• Who do you want to use the tool and workflows?
  • Individuals? Teams? Are you sharing across teams? If so, how many?
• What data sources are you interested in, and where do you want the data to go?
• When do you want the data prep workflow to run?
  • Scheduled? When you push a button? Run from an API?
• Where do you want the tool to run?
  • On premises? On the cloud? On your local laptop?

This is by no means an exhaustive list, but it’s a starting point.

We’re going to delve into the tools now and take a look at each of them working on the same process and how they each arrive at the desired outcome:

We’ll be working through the tools as we have seen them work with clients. In this instance, we will run through the tools as if we were a bank working with two different data sources:

• All Loan Requests – What the loan request entails, an ID for customer, amount, how long it’s for and other credit agreements.
• Customer Information – The personal details on the clients

The goal is to bring the information together and conduct data prep on these files before importing into Tableau.

Scenario 1 – Tableau Prep

Tableau Prep is Tableau’s own tool, so if you are a Tableau user, the interface will be extremely familiar and is just as intuitive as the tool itself. Tableau Prep, like Tableau, has so many native APIs that you can select to connect your data. We love this feature.

Step 1 – Bring in the data sources:

Step 2 – Review the data imported in the summary screen:

Step 3 – Perform calculations:

In this instance, we can see that Date of Birth will come into Tableau a little messy, so we are going to use DOB to calculate an actual age for our customers.

Step 4– DOB calculation:

You’ll recognize this calculation field if you are currently a Tableau user. Essentially, you’re taking the calculation out of Tableau Desktop and doing that work in Tableau Prep, so when it comes to using Tableau Desktop, that work has already been done:

Now that we have the parsed field, we can create a calculation looking at DOB from today’s date to calculate an individual’s age:

All of the prep you have completed is easily identified on the screen, so if you want to make alterations or just have a record of what work has been done, it is available for you to review:

The final step in the data prep is to remove data you don’t wish to carry through. Taking all of the data into Tableau Desktop could lead to the performance lag of your visualization, so it’s better to remove it prior to the import:

Step 4 – Join:

Finally, we create the join. In good, old Tableau fashion, this is a simple drag-and-drop action and selecting the type of join. Due to the ID column, this is done efficiently. You can then output the data and decide the type of file you wish to export to.

Scenario 2 – Dataiku

Let’s take a step-by-step walk through of exactly the same actions, but this time, we’ll perform them in the cloud-based Dataiku tool.

Step 1 – Import the datasets.

Step 2 – Similar to Tableau, you have an array of native APIs to choose from:

Step 3 – Select your recipe:

Tableau calls these “steps” but in the Dataiku tool, we have recipes. The Visual recipes are pre-written for you, Code recipes allow you to write your own code, and Plugin recipes are external sourcing you can use. We are going to be working with the Visual recipe Prepare.

Step 4 – Clean your data with calculations:

Dataiku is prompting you to parse the data, goes on to show you the recommended format and identifies how much of your data will be impacted by the parse.

Once again, Dataiku has recorded all of the work that has been done to complete the last mile so that you can review:

Step 4 – Joins

Finally, using the visual recipe card, you want to complete the join:

Step 5 – Export:

Scenario 3 – Alteryx

Finally, let’s move onto Alteryx. Where Tableau and Dataiku like to group things together, Alteryx requires you to do these steps individually and has a far more detailed workflow on the canvas.

Step 1 – Parsing the DOB:

Step 2 – Create your calculation:

Step 3 – Create your formulas:

Step 4 – Join your two datasets:

Here is your final, detailed data prep:

And here is a look at the workflows by tool:

Next Steps After Data Prep

Now that we have prepped our data, what do we do next?

Immediate Success!

• Get to your answers quicker – you no longer have to ask your IT team to support any data changes/fixes.
• Visualize things quickly and effectively – you’ve done all of the hard work before you bring this data into Tableau.

Future Success!

• Data Science
  • Prediction, classification, customer segmentation
  • Native options within Dataiku and Alteryx

So, which tool is right for you? It really depends on your immediate actions/requirements, as well as what you see yourself doing with your data in the future. If you want support reviewing those considerations or want to discuss the different tools with a guide who is well versed in them all, feel free to contact us! We’d love to help you make the best choice for your use case. And check out the webinar recordings below to watch the full walkthrough of the scenarios above.

Webinar Replay – US 

Webinar Replay – APAC

Webinar Replay – EMEA 

The post Webinar Replay: The Data Prep Landscape appeared first on InterWorks.

Subtract the rush of busyness and the freedom to gather with other people. Add an overload of technology during the day and a global pandemic. What do you get? A lot of time on your hands and a desire to return to simpler things. This installment of our hobbies blog series presents: cross-stitching.

A Way to Slow Down and Be Present

What is one hobby you really enjoy?

Karlee: Lauren Roberson taught me to cross-stitch as a way to relax and have a hobby not involving screen time. I am so thankful for a mentor who knew exactly what I needed!

Rachel: Cross-stitching. I’m currently cross-stitching something I’ll be framing for a friend of mine who just recently had their first child.

Above: Karlee’s big box of cross-stitching supplies

What do you find so fulfilling or rewarding about this hobby?

K: Cross-stitching is rewarding because it takes a lot of concentration and time. It is fun to see a project come together knowing you created the entire thing. You can also create anything – witty, funny, encouraging, art, you name it!

R: There’s something to be said for creating something. It’s very therapeutic. Also, this is a hobby that allows me to listen to things at the same time. I’ll listen to a podcast I’ve been meaning to or watch a documentary (I just binged Wild, Wild Country this weekend while working on this).

Above: Rachel’s cross-stitching project

Resetting Expectations for Yourself

How is this hobby helping you during this time of quarantine and social distancing?

K: It keeps me distracted! I am an extroverted person, so being in the house 24/7 is tough! It also is nice because I can make something for other people and send it to them in a care package to let them know I care for them (type 2 Enneagram over here!).

R: I’m sure I’m not alone in feeling unsettled during this time. There are so many unknowns, including, but not limited to, when can we stop social distancing and quarantining, and when can life get back to “normal”? Cross-stitching can be a very slow process. There are times I have the feeling of “I’ll never finish this. It’s never-ending. I’ve spent 30 minutes here and have finished two leaves.” But then I’ll step back and realize that I’ve gotten further than I thought. Also, I’ve been feeling very disconnected. I’m an extrovert by nature, and this has definitely been a trying time for me. Doing this cross-stitch project as a gift for someone is helping ease that a bit.

How does this hobby help you in your day-to-day role at InterWorks?

K: From day to day, it helps me turn my brain off. I am super goal-driven, so having 24/7 access to my computer can be hard because I just won’t stop working! This hobby is nice because I can’t think about who I need to email or who I need to take care of when I do it. Plus, it is so calming that I usually fall right to sleep later!

R: It has helped me to realize that to stay efficient, I should set smaller goals and to celebrate when I meet these small goals. Finished a leaf? That’s awesome! Finished a color? Look at you go!

Above: Karlee’s first ever cross-stitching project

Finding Balance and Rest

Why is it important to have hobbies/personal projects?

K: I think everyone should have something they love. And you can have multiple things! I love golf, going to the farm, baking, working out and more! It’s important to do something you enjoy to help you realize what is important and let your brain relax. We put ourselves under so much stress every single day. Having something where you can turn off the screen and just be present is crucial.

R: We all need a distraction right now. Being glued to the news is not good for my mental health, and I’m guessing the same can be said for many. We need to stay informed, but 24-hours a day can begin to do more harm than good. Focusing on something else can help bring you back to balance.

What has this taught you about yourself? What can hobbies in general teach us about ourselves?

K: Cross-stitching has taught me how to be patient with projects. You don’t always get instant gratification, but it is so fulfilling to see a project come to life. It also has helped me realize that I don’t need to check my email all night long – because being present matters!

R: Hobbies in general can teach us what we need to recharge. People always talk about how extroverts vs. introverts recharge, and that’s always centered around people. Extroverts recharge by interacting with people; introverts recharge by taking some alone time. However, for some, those ways of recharging have been removed. Extroverts who live by themselves can’t interact with people, and introverts who have roommates they are quarantined with have more difficulty getting the alone time they need. So hobbies can serve as the replacement. Do you recharge by learning something new? Creating something visual? Composing music? Exercising? Whatever it is, go do it!

Above: The beginnings of a new project for Rachel

The post Out of Office: My Cross-Stitching Hobby appeared first on InterWorks.

Everyone has that moment. You’ve got the data. You have the goal – make a dashboard/visualization that says something interesting. You sit down. And then you freeze. You’re staring at a blank screen. Where do I go from here?!?!? The possibilities are virtually endless and it’s terrifying. Sometimes, it’s feeling paralyzed by all the options. Sometimes, it’s feeling the intense impostor syndrome that so many of us face. But whatever it is, it’s debilitating. Generally, people just see the finished product. A beautiful dashboard that’s polished and shiny and whole. What people don’t see is the process—the often stressful, sometimes agonizing, existential-dread-inducing process—leading up to it.

Where Do I Even Start?

To combat this, I have created a process (with steps I can check off because who doesn’t love crossing off something on a to-do list?). Hopefully, it can help you as well! Get your pen and paper out and let’s get started! (Actual pen and paper. I do a lot of these steps in a physical notebook)

For each of the steps, I’ll show my process through building this dashboard for Week 7 2019 of Makeover Monday (find the data here and the Makeover Monday website here).

Step 1. Write Down All the Questions

At this point in the process, I want you to think of the data very theoretically. What does your data involve? Is it sales data? HR data? Test scores? Survey responses? Whatever it is, think about some of the questions you might want to answer with it. Write every single one of them down:

Step 2. Write out the Charts That Go with Each Question

Now that you have a list of all the questions, write out charts that would be helpful in visualizing the answers to the questions. Some of the questions might have multiple charts that would work. For example, if you are an online store with sales data, and you want to see what department the majority of your orders belong to, you could possibly show that with a pie chart, a bar chart or even just a BAN (Big A$$ Number) type visualization where you only show the name of the top department:

Step 3. Group the Questions That Go Together to Create a Story

Up to this point, you’ve let the creative juices flow and have written down almost every idea that’s crossed your mind. This is the part where you’ll start taking a more critical eye to your ideas. Creating a visualization that answers every question you wrote down would be a bit overwhelming for the viewer. To help narrow down the questions, begin to group them together. If you already have a story you want to tell in mind, what questions help? If you don’t have a story already, which of your questions go together? Grouping them will help you come up with a cohesive story.

Step 3a. Get the Data You Need for Your Story

A sub-step of this is to begin to look at the actual data and figure out if you have all the data you need to tell the story you’ve chosen. It might be that you need to collect or acquire more data. This is also the time where you might need to create calculations to get the data you need. For example, say you want to look at % towards a goal instead of the raw number. That number may not be in your data as is, but if you have the Target and the Actual numbers, you can create a calculation to determine it:

Above: This is a politically charged dataset, so I wanted to make sure the story I told was simply reporting the data without spin.

Step 4. Make All the Charts … Yes, All the Charts

You should now have a slightly smaller list of questions. For each of these questions, you have at least one chart idea. Now, you should make them all—every single one of them. You don’t need to change a lot of the formatting such as the color and the font. Just build the bare-bones version of each of the charts. Why? Building all the charts helps you see with your very eyes which is the best way to visualize the answer to your questions:

Above: This is a cleaned-up look at all the charts I created in Tableau.

Step 5. Draw out What It Could Look Like

Now we go back to pen and paper (or whiteboard and marker) to make our blueprint for the dashboard. Usually, I’ll have my paper represent the dashboard and then using a pencil I’ll start to place and draw my visualization. This will go through many rounds, hence the use of a pencil. Below is a sped-up video of my process for this project:

Step 6. Create

Put the dashboard together to match your drawing. For me, this part usually takes the longest because this is the time when you’ll edit the formatting to make everything cohesive. The amount of time I’ll spend choosing fonts would astound you. If you want, try to set a time limit on this portion:

Above: My first attempt at the dashboard I drew out (after a lot of formatting)

Step 7. Ask for Help (Optional but Recommended)

Reach out to ask for feedback on what you’ve created. I generally use colleagues I have at InterWorks. If you do not have that community, the Tableau Twitter community is very helpful. To me, this is the most nerve-wracking part: sending something I’ve created to someone else for critique. It’s the time when impostor syndrome starts to kick in. “What if they think it’s awful? What if they would have done it completely differently and are now judging me and will never trust my work again?” Try your hardest to push past these thoughts and reach out to your community. Remember—your community is here to help you, not bring you down.

Getting feedback and practicing this is the best way to get better at dashboarding, so the sooner you can get comfortable baring your soul (or so it feels) for feedback, the better:

Above: The comments I received back from my colleague Keith

Step 8. Walk Away

Have you ever been working on a puzzle and are looking for a particular piece and even though you swear you’ve tried every piece, you just can’t find it? Then you stomp off in frustration. A few hours later, you come back, and the very first piece you look at is the piece you were looking for all the time. This step is basically that same idea. You need to give your brain a chance to reset, which will ultimately help you make a better product.

Sometimes this is the most difficult part. A lot of times, I’ll combine this with Step 7 as a way of forcing myself to walk away. When I hand it off to someone to look at and give their opinion, I vow not to edit it until they respond.

Step 9. Come Back and Move Things Around

Now that you’ve had a breather and have probably received some feedback from whomever you reached out to in Step 7, it’s time to get back into it. Move things around, remove things, edit the things that are there, iterate.

Step 10. Repeat Steps 7-9 Until You’re Satisfied (or Satisfied Enough)

One sentence I keep in my mind as I’m working on dashboard is from Voltaire: “Perfect is the enemy of good.” Or to put it another way from Confucius: “Better a diamond with a flaw than a pebble without one.” You could spend forever tweaking the dashboard and not feeling like it’s good enough, but remember, this is already better than what you had to start with! My final dashboard can be found below.

These are the steps that have worked for me to get over that blank page debilitation, and I hope they help you, too! I’ll leave you with one last quote, this time from Zig Ziglar: “You don’t have to be great to start, but you have to start to be great.” You got this!

The post Dashboard Confessions: Fighting Blank-Page Paralysis appeared first on InterWorks.

As you may have heard, InterWorks’ best asset is by far our people. It’s the thing I’m thankful for day in and day out and one of the first answers I give when asked why I enjoy working here. Today is International Women’s Day, and here at InterWorks, while we celebrate the amazing women that work with us every day, we thought we’d take the opportunity to celebrate and highlight some of our people a little extra today.

A Data Viz for International Women’s Day

What better way to kick that off than with a viz of the women of InterWorks? We’ve got some awesome women that work across every part of our business all over the world. Hover over individuals to see the role they play within our organization, as well as where they’re located. If that’s not enough, you can click on each of the pictures to read more about individuals in their bio, as well as see some fun hobbies and activities in the photos as well. You can also head over to our People page to see all of our employees and their bios. (Find the viz at the bottom of this post.)

Next, we reached out on Slack and asked folks to respond to a few questions:

• What’s your favorite part of working at InterWorks?
• How did you get into your field? Do you have any advice for other women looking to get into your field?
• What women are you celebrating on International Women’s Day?

For me, I know I’ll be celebrating all these amazing women, as well as those who have been my teachers and classmates throughout life. But specifically, I celebrate my mother and my sister who daily demonstrate what it means to be resilient and steadfast in all facets of their lives.

Below are some of the responses from our team. We hope you enjoy, and as you’re thinking about these questions, we’d love to hear from you about the women you’re inspired by and celebrating today and every day!

Above: The women of the Employee Experience team at a team meetup

Debbie Yu | Analytics Consultant 

What’s your favorite part of working at InterWorks?

Making strong connections and friendships with very genuine people. One would think that in working at a company where a lot of the workforce is remote, it would be hard to create meaningful friendships, but it’s been the opposite. I’m forever grateful for the friendships I’ve made here, and I know they will be long lasting.

Do you have any advice for other women looking to get into your field? How did you get into it?

The great thing about analytics is that there are a multitude of paths to take! The story for a lot of people in data and analytics is that they started out being the go-to person for providing analysis and solutions through data, whether it was their job function or not, and then they just kept expanding their skillset and exposure on their own until finding a role in analytics. I was working as a data analyst in education and found myself wanting to expand my skillset beyond Excel, so I took a data science part-time course and moved into the private sector for more experience. If you’re interested in data—there are SO many resources out there (cheap and low cost, too)! Take a class, go to a meetup or a women-in-data/tech event and volunteer at hackathons. This will help you understand if that’s what you’re really looking for, and by meeting people in various roles, you can think about how you want to evolve your own skillset for one of those roles someday. And most of all, don’t be intimidated! You can do more than you’ll have ever imagined if you just keep going and be persistent.

What women are you celebrating on International Women’s Day?

SO many women!! I’m not the woman I am today without the support from all the women in my life—starting out with my mom, who came to the US in the late ‘70s with a dream to build an amazing life for her family; my childhood girlfriends who I’ve kept in touch with for 20+ years; and women I’ve met along my professional journey who started out as colleagues and are now lifelong friends and mentors.

Above: Carly, Katie and Debbie at the 2018 Holiday Party

Sarah Dorfman | Analytics Consultant 

What’s your favorite part of working at InterWorks?

It feels great to work at a place where your passions and ideas are encouraged to come to fruition, and it’s even better I get to work with people with similar enthusiasm.

Do you have any advice for other women looking to get into your field? How did you get into it?

My journey into analytics was a bit roundabout, but the common thread in my route here was my interest in problem solving and understanding and helping others. I was a Japanese major in college, but living in another country and culture sparked my desire to learn more about marketing and consumer behavior, which led to graduate studies in this area. As part of grad school, I began to use software like SPSS (statistical analysis software), which helped me get a job on an analytics team at an ad agency. With each job thereafter, I added to my technical skillset with SQL and Tableau. I also consider myself lucky to have had bosses who recognized my capacity and excitement for learning! With that in mind, if you’re looking to move into analytics, and you don’t have a big technical background, I’d recommend taking some online classes on sites like Udemy to get familiar with tools and concepts and then look for companies and managers that embrace and support learning.

What women are you celebrating on International Women’s Day?

My mom, my aunt and my grandmothers. All of them have worked incredibly hard in a myriad of careers. They’ve worked hard so they could make life better for our family, including helping me pursue higher education, and for that, I am incredibly grateful.

Above: A Stillwater office painting meetup

Sian Davies | Analytics Consultant 

What’s your favorite part of working at InterWorks?

My favourite thing about working here is how creative I get to be. I wouldn’t normally call myself a creative person, but I am constantly creating things here: content, ideas, presentations, courses, pitches, reasons to talk to people. This means I’m never bored, and there’s a big chunk of my brain being stimulated.

Do you have any advice for other women looking to get into your field? How did you get into it?

Don’t even start to believe that this isn’t a field for you. Times have changed since your mum was a lass. There are more opportunities for women now in this industry than there have ever been (i.e. girls are learning to code from age four), and there is nothing in your chromosomes that says you can’t be a tech entrepreneur, a developer, a consultant, a CTO. Just kick the door down if it’s in your way, because it’s only made of paper.

I got into this industry through banking. I don’t come from a tech background at all, nor a finance one, but I arrived at a place in my life where I needed a good income and got hired on my gumption. I jumped in and taught myself, following a meandering path through consulting, self-employment and start-ups to get me here.

What women are you celebrating on International Women’s Day?

I have decided to celebrate women on international women’s day who aren’t famous to the world’s eyes but who have broken through their own personal glass ceiling and inspired me. I need to give my mum some love for being a woman from South Wales who has no university education, came from a miner’s family, learned to rally drive when she was 15 and pushed her two daughters into a top school and through to post-grad level. All of this was done to the shock of many of her peers and relatives, I think. She hiked Machu Picchu on her 50^th birthday, rode the NY coaster in Las Vegas on her 60^th and learned to ski in the decade after that. She’s recently learnt to ballroom dance though she has trouble following a man’s lead.

Other women I will be celebrating are Inge and Dion, who choose to live in Georgia, a country where women are still largely expected to follow men, be quiet and stay at home to bring up children. Both are successful in their respective freelance careers while also being feminists, parents and incredibly stylish.

Above: Sian and Helen surfing

Katie Bridges | Analytics Consultant 

What’s your favorite part of working at InterWorks?

I love that there’s always an opportunity to challenge myself and that I’m surrounded by other people who are incredibly encouraging and working hard to create and deliver awesome work.

Do you have any advice for other women looking to get into your field? How did you get into it?

Through a series of random connections and events and quite a bit of work and learning. It was a roundabout journey, and I think it’s so different for everyone. I’ve always been a very tactile and hands-on learner, so I learned a lot (and still do!) by finding things I enjoy or care about and incorporating those into a new skill or tool. I personally think it’s way more fun to show off something you’re proud of than a random online exercise.

My advice is to trust yourself. We were all beginners at some point, and it’s easy to let that discourage you from learning something. Don’t be afraid to make mistakes or take on something new.

What women are you celebrating on International Women’s Day?

So many! I feel so fortunate to have so many inspiring women in my life from family to friends and colleagues. This year, I’m also especially celebratory of women being authentically themselves and constantly reminding us to continue working hard and pushing forward. To see this happening in so many spheres—entertainment, politics and tech, just to name a few—is so encouraging.

Above: Pam and Chelsie showing their OSU spirit at the office

Andrea Avey | Content Coordinator 

What’s your favorite part of working at InterWorks?

My favorite part of being at InterWorks is being surrounded by such talented, warm and generous people every day. Everyone I interact with is always willing to help with any questions I encounter, no matter what they may be in the middle of, and we have a culture that makes people feel valued, known and celebrated.

Do you have any advice for other women looking to get into your field? How did you get into it?

No part of my professional journey has been what I expected. I studied English and Spanish in college, and after an unexpected stint in education, I now find myself working for a tech company. The only thing that has remained consistent along this path is my process of trial and error. I was looking for a strong and rewarding company culture that would allow me to use my skills and continue to grow and be challenged when I found InterWorks. I think it’s crucial to try things out with an open mind. Honestly examine what you want, and once you figure it out, don’t let anything stand in the way of your pursuit of it. Trust the process and trust yourself.

What women are you celebrating on International Women’s Day?

I am full of gratitude for my mom each moment: a woman who has always commanded respect and maintained her strong will, while remaining humble and approachable. I will be satisfied if I end up with a fragment of her fire and wisdom. To be frank, I’m feeling pretty awed by the women I work alongside at InterWorks. Each of them is incredible in her own right and deserves to be celebrated daily.

Above: Rachel and Liz at the horsetrack in Saratoga Springs during a team meetup

Lyah Barberan | Project Coordinator 

What’s your favorite part of working at InterWorks?

My favorite part of working at InterWorks is the family dynamic. I know I can rely and count on everyone here and that everyone has each other’s backs. On top of that, everyone here is an absolute rock star at what they do. I’ve never met such a large group of talented, intelligent people. InterWorks folks radiate good energy; any time I’m in an office or in a room with my IW family, my heart is just filled with happiness and love. I seriously love these people.

Do you have any advice for other women looking to get into your field? How did you get into it?

I got to grow up with technology as it grew itself. When I was young, I would watch Disney movies on VHS tapes and play computer games when the internet was still dial up, and Instagram wasn’t a thing. My love of technology was always relevant to my life, and I eventually wanted to know more. I always had to have the newest, latest, greatest piece of tech, and that has carried on into my adulthood. It really wasn’t until high school that I figured this could really be my jam. I started to research what the IT industry was all about. It didn’t take long for me to notice that this was not a common field for women. I thought, “Well, let’s change that.” If there are any women that want to get into IT, my advice is to do as much research as you want and find the areas that interest you most. Don’t be afraid to be the only woman in the room, and don’t be afraid to speak up and make the right kind of noise. People are often surprised when they learn what industry I work in—hey, girls can be into computers, too! Break the stigma and set the expectation of being great. Women are the most powerful, especially when we are working toward something that we are passionate about. The resources are out there and available to you; you just have to find them.

What women are you celebrating on International Women’s Day?

Today, I’m celebrating all women. A few that inspire me on a daily basis are Serena Williams, Katherine Johnson and Michelle Obama—women that are strong, smart, kind and all-around badass. They serve as a reminder that I can do anything that I put my mind to and to never sell myself short on success.

Note: This blog and the data viz below was created collaboratively by Rachel Kurtz and Katie Bridges.