Democratizing Data Apps with Streamlit

The internet has changed a lot in the last few decades and so has the way we create web and mobile applications. Today with Web 3.0, we enjoy a smarter, more decentralized, and secure online experience that has opened new possibilities by enabling innovative business models and applications for industries, such as finance, healthcare, and e-commerce.

The Evolution of Web Applications with Web 3.0

And while data apps are different from web apps in what they do and how they work, they too have benefitted from technology evolutions like Web 3.0. It has made the development process much easier for data app developers since they can now focus on solving business problems instead of worrying about things like the app environment, deployment, scalability, etc.

Figure1. Source: https://blog.sketchar.io/what-advantages-does-web-3-0-give-to-creators/  

What Are Data Apps?

Data apps are powerful tools that enable users to unlock the full potential of their data. They allow users to explore, transform, visualize, model, and report data in ways which help users discover hidden patterns, optimize data workflows, create stunning data visualizations, build data models, and generate data reports. Data apps are an integral part of the modern data ecosystem as they empower individuals to make informed, data-driven decisions. Data apps are designed to handle, process, and analyze data in different ways, such as data visualization, data analysis, and data storage.

Streamlit and Snowpark for Building Data Apps

Streamlit is a great choice for anyone looking to create data-driven web apps with Python.  It’s powerful, flexible, and easy to use. And thanks to its integration with Snowflake, you can build and scale your data apps with ease.

With Streamlit developers need to write minimal code to build powerful applications and with Snowpark developers get an API which they can use to construct DataFrames that are executed lazily on Snowflake’s platform. With Streamlit and Snowpark combinations, developers can build powerful Snowflake applications with very little coding.

With Streamlit, one can write simple code to create interactive components that use DataFrames as the data model. One need not worry about web development or deployment details. One can write a Python file and run it with Streamlit.

As a developer myself, I love Streamlit for its simplicity and intuitiveness. Even if you don’t know Python or are new to it, you can learn Streamlit quickly and focus on the core logic of your app. And with Streamlit cloud, you can deploy your app to the web without any hassle. If you have ever used other cloud platforms, you know how much time and effort they can save you.

Figure2. Source: https://streamlit.io/

If you are planning to build a data app, follow these steps to get started. 

Getting Started with Streamlit and Snowflake 

How to connect Streamlit to Snowflake

Step-1: Create a Snowflake database by logging into your Snowflake account. (If you don’t have a Snowflake account then sign-up for a 30-day free trial account at https://signup.snowflake.com/)  

Step-2: Setup a Conda Coding environment.

First, create a conda environment called snowpark by entering the following into a terminal (also referred to as the command line) window

conda create -n snowpark python=3.8

Once everything is all setup and ready to go, there will be a message at the bottom with instructions on how to activate and deactivate the environment.

conda activate snowpark

Next, we’ll want to install some prerequisite Python libraries so that we can build a Streamlit app that can connect with a Snowflake database.

pip install snowflake-snowpark-python streamlit

This will install the Python libraries as required. For more information, refer to https://conda.io/ and also feel free to use any other similar package management system of your choice.

Step-3: Setup directory and files:

Create 2 empty files for our Streamlit app that we will cover in more depth in the next section:

touch streamlit_app.py
mkdir .streamlit
touch .streamlit/secrets.toml

Step-4: Building the Streamlit app:

Populate the required code in the streamlit_app.py file

import streamlit as st
from snowflake.snowpark import Session
st.title('Learning how to connect Streamlit to a Snowflake database')
………..
………..

Step-5: Adding access credentials via secrets management:

Remember the .streamlit/secrets.toml file that we created earlier, we’re now going to populate that file with the following content:

[snowflake]
user = "enter_your_password_here"
password = "enter_your_password_here"
account = "enter_your_account_key_here"

Step-6: Launch the Streamlit App

streamlit run streamlit_app.py

you should be able to see a new internet browser launching with the Streamlit app that is running locally.

Optimize Your Data App

As you move your Streamlit app from the development or analysis stage to production, it is critical to optimize the performance of the app so that there are no outages and users get the best experience.

Based on my own experience, the below items help:

• Avoid downloading large and static models – As you push your Streamlit app to the production environment, bring your model and other assets to the production machine, and you can get orders-of-magnitude better startup time.
• Pre-calculate inputs – When moving a Streamlit app to production, consider whether there are parts of the app that can be pre-calculated. If the data in question never changes, you can do this calculation beforehand outside of the app.
• Caching – Caching stores the results of slow function calls, so they only need to run once. This makes your app much faster and helps with persisting objects across re-runs.
• Garbage Collector – Remove unused data. It is highly recommended to remove unnecessary columns and rows you aren’t using. Your data will get read in faster, use less RAM, and overall be more efficient when calculations are performed.
• Optimize data storage formats – By using an optimized data storage format, your production app won’t spend time parsing text into data types such as integers, floats, and strings, which incredibly, can consume quite a bit of time. Additionally, binary formats often have metadata and logical partitioning such that Python can read the metadata to find exactly where the data are located, skipping entire data partitions from loading.

Key Use Cases of a Streamlit-based App

• Machine Learning Apps

  – These apps are built for key stakeholders so that they can run your machine learning model through an easy-to-use intuitive interface instead of some code they don’t understand.

• Visualization Apps

  – It is one of the key use cases and there are so many Streamlit-based apps built primarily for visualization. Since it doesn’t require any knowledge of web design tools such as HTML and CSS (unlike Flask), it has become extremely easy for developers. Feel free to check out a few apps at https://streamlit.io/gallery

• Apps for ML Ops

  – Being a production-ready app framework, Streamlit offers the fastest way to build apps for machine learning models. Streamlit Cloud workspace helps manage apps and ensures direct app launching through the user’s GitHub repository code.  It provides a seamless experience in app development stages – layout, input, output, call backs, and interaction. It makes it a good playground for ML Ops use cases.

• Data Science Apps

  – For Data Scientists, Streamlit can help them load, explore, visualize and interact with data, and generate dashboards in less than 100 lines of Python code.

In Summary

Streamlit is a powerful and easy-to-use web framework for Python. It lets you create interactive web applications and stunning data apps with just a few lines of code. You can also deploy your app to the Streamlit cloud in seconds and scale it effortlessly. And with Snowpark and Snowflake integration, you can leverage all the best tools for data engineering.

So, why wait? Turn your idea into a data app with Streamlit today! 

References:

https://signup.snowflake.com/

https://quickstarts.snowflake.com/guide/getting_started_with_snowpark_for_python_streamlit/index.html?index=..%2F..index#0

https://blog.streamlit.io/six-tips-for-improving-your-streamlit-app-performance/

https://blog.streamlit.io/introducing-streamlit-cloud/