There are many different services on Azure that are used for IIoT solutions. Playfully I call this versatile stack as a jungle. And why not. We can see few different layers there from usage and also from architectural perspective. But before we go into details let’s take a step back to see the forest from the trees so to speak. Download our blueprint about IIoT.
I 💙 pandas, I really do. And I also like to do data profiling. So I can’t possibly go wrong with panda profiling, right? Well, jokes aside, in my opinion data profiling should, at least to some extent, be a standard practice in all data processing work. For me and for many of my colleagues our history with data development goes far further than to the time before we had all these nice tools we have now, though. The process of checking data issues has previously been mainly manual. Some of us who are lazy enough...
Microsoft Power Platform is a cloud-based set of applications that allow companies to automate processes and build custom applications, visualize data and distribute reports. It brings low-code/no-code development and business intelligence tools to business users.
A week after I started to write the first blog post covering Great Expectations framework, I am back at it again. I managed to first create a custom expectation (i.e., a custom data validation rule) and after which I investigated the more formal way of using the framework. Here’s how it went and what I learned.
Machine learning (ML) applications are already used in many businesses, especially in data or analytics-heavy industries. Whether small or large, every organization can have a lot to gain if they decide to utilize advanced analytics. For example, using an optimized supply chain will help decrease costs. Clustering customers will help to target marketing departments' efforts to the right customer group. Classifying customer churn will help target efforts to stop customers who are about to leave, which results in...
So, here’s the deal. One weekend I found myself stuck at home in Covid quarantine waiting for the test results for my kid. Instead of watching the TV the entire Sunday, I decided I might try to use my time doing something a bit more productive.
If you follow any trends in data world, you have probably heard about Data Lakehouse. It’s a new architecture proposed by Databricks that utilises Delta tables as data lake storage format.
In a previous blog post I talked about how Databricks Data Lakehouse can be created with low code implementation only. That is almost true. System needs to be setup and for that initial configuration some code is needed. What this code does is it creates a mount to storage account that will be used as storage for delta tables. Fortunately this code is well documented and there are multiple guides to accomplish this like this one:...
Organisatoric data and reporting secrets revealed Where does this data actually come from?
tl;dr Came here just for the script? You can find it here: https://dev.azure.com/cloud1public/_git/Blog%20Source
