How to efficiently interact with config files in Python — Originally published at https://mathdatasimplified.com on May 26, 2023. Problem In your data science project, certain values tend to change frequently, such as file names, selected features, train-test split ratio, and hyperparameters for your model.

Discover the Hidden Benefits and Drawbacks of dbt — Motivation If your organization is looking to create a data-driven product, you should consider having efficient data pipelines to: Stay competitive: Quick access to data and its analysis through efficient data pipelines accelerates decision-making and keeps you ahead of your competition. Reduce costs: The time and effort required to collect…

Faster Time to Market and Increase Efficiency — Motivation Consider this scenario: A more accurate machine learning model is developed every month and added to the main branch. To deploy the model, you must download it to your machine, package it, and deploy it.

Speed Up Iteration and Cut Computation Cost — Motivation Many data teams begin by establishing an analytics practice on data warehouses such as BigQuery. However, solely relying on BigQuery for data science workloads may not be the best approach due to various reasons: Advanced needs beyond SQL: Use cases such as data validation, visualization, and machine learning forecasting may…

Automate Machine Learning Workflow with Continuous Integration — Scenario As a data scientist, you are responsible for improving the model currently in production. After spending months fine-tuning the model, you discover one with greater accuracy than the original. Excited by your breakthrough, you create a pull request to merge your model into the main branch. …

Use natural language to test the behavior of your ML models — Motivation Imagine you create an ML model to predict customer sentiment based on reviews. Upon deploying it, you realize that the model incorrectly labels certain positive reviews as negative when they’re rephrased using negative words. This is just one example of how an extremely accurate ML model can fail without…

Start Your Next ML Project With This Template — Motivation Getting started is often the most challenging part when building ML projects. How should you structure your repository? Which standards should you follow? Will your teammates be able to reproduce the results of your experimentations? Instead of trying to find an ideal repository structure, wouldn’t it be…

Incorporate Domain Knowledge into Your Model with Rule-Based Learning — Motivation You are given a labeled dataset and assigned to predict a new one. What would you do? The first approach that you probably try is to train a machine learning model to find rules for labeling new data. This is convenient, but it is challenging to know why the…

Create a UI for ML Feature Engineering in One Line of Code — Motivation As a data scientist, you might frequently adjust your feature engineering process and tune your machine learning models to get a good result. Instead of digging into your code to change function parameters: …, wouldn’t it be nice if you could change the parameter values from…