Impact 2024: The Industrial Data and AI Conference for and by Users | Nominate Speakers Now for a Ch...
Hei @nabanitakar Thanks again for reaching out regarding this. With regards to AIR it is being matured as a more robust and scalable service within CDF the first iteration of which will be through Charts. With regards to your use case i would recommend writing a cognite function that generates a time series as output based on the out put of your forecasting model on a schedule say 5 mins. I would then read that time series in charts and setup a monitoring job.However please note that monitoring and alerting is still in beta and only avaailble for early adopter customers so i would recommend reaching out your customer success person to help with setting this up for your project.
Hei thanks again for your feedback on this ! Yes it should alert in that situation. Basically that parameter is just the threshold value and the backend checks if an alert lasts for atleast one minute and then sends out an alert. I will ask the team to look into this on the backend side to make sure things are in order. Do you have the correct CDF groups setup ?
Hello thanks again for this feedback on the monitoring solution called AIR that we have today. We have been working on maturing AIR into a more mature product that is a part of CDF and one of the major improvements is the latency time between alerts and the emails being sent out. So stay tuned for more updates!
Yup that makes sense ! this is something thats already a part of the new monitoring in charts solution we are building so we hope that will solve this problem in the long run.
Great feedback on this ! Would love to get more of an understanding but would this be related to having an endpoint way of subscribing to alerts ?
Cognite Charts has a good features that helps in analyzing time series data. One suggestion related to the Threshold feature is to add the ability to trigger notifications based on threshold exceedance. The notification should be customizable to avoid too many notifications. In addition, it would be great also to provide more options for the types of notifications. Example, Email, SMS or popup. This feature will be very useful for, domain experts, operators and engineers to setup an offline monitoring task that can notify them to go back to CDF whenever a notification is triggered to analyse. I think it will increase Cognite Charts utilization by making it a great source of root cause analysis whenever a notification is triggered. Hei Adeep thanks again for your great insights on this. On a very positive note the team is currently working on this and you will be able to set monitoring on time series within charts on thresholds with an ability to receive email notifications. So stay t
This is a fantastic use case for AIR but how would we do this if you decouple python modelling from AIR. Thanks again for the great question @ibrahim.alsyed . The way i would recommend working with this use case is to write a function in cognite functions that writes a KPI and then you could select the thresholds to be applied on the KPI based on your use case. One of goals in the future is to make this process much easier to do and reduce the time from KPI generation using user generated code to value.
It would be events. Yeah if this is the case then yeah its something we will start looking into as we come upon a more stable experience for time series alerting in CDF both feature wise and performance wise in the months to come. While we are working on this we will reach out to better understand these use cases and the flows that go into such a event driven alerting system.
Can we do unsupervised Machine Learning Anomaly Detection on Time Series Trends in AIR? This is a bit hard to answer without knowing the details of the models and the inputs that go into these models. With the alerting approach that we going with today the way you could achieve this is by writing a cognite function that runs a pretrained model and writes some kind of KPI. You could then potentially do alerting on these as we expand on more thresholding capabilities. Would be a cool exercise for us to explore these use cases together and figure out what he missing pieces are. I see strong potential in refining that cognite function experience as we get more user input and providing less friction between deploying these models and setting up alerting.
Is Alerting on possible using Time Series data? What if i want alerting on transaction such as Work Orders or Product Recipes that change etc? So alerting as we are working on today is focused on a time series approach and providing a stable and scalable experience not only across numbers but across use cases. Although alerting on events is not something that is coming soon, what we are working on is to enable the infrastructure to allow this to be possible in a scalable manner while we understand the use cases and workflows that into such a event driven system. We are hoping to reach out later in the year to our users as we move towards a more stable product to understand and solve this use cases better so stay tuned. I do wonder when you say work orders and product recipes what would a representation of that look like in CDF ?
@Arun Arunachalam and @Knut Vidvei - The one thing we feel is missing from this list that would make alerting far more valuable to end users (and differentiating from other alerts we already have on our facilities) is the ability to generate an alert workflow given we rarely rely on a single alert to take action. Example: if we’re monitoring the flow through a pump and we want to be alerted when the pump is encountering issues performing, we expect the flow to go to zero when the entire facility is down. The only time this alert would be of value is if the facility is online and the pump rate drops below a specified threshold. Example 2 - most of our RCA’s involve some sort of series of events that are currently captured as RCA’s that are static rather than building it in to a monitoring system. By allowing for this workflow to be “automated” within AIR, we can now ensure that these events are being monitored to be prevented in the future and it would be beneficial to pull up as part
@ibrahim.alsyed yes python will continue to be a part of our infrastructure but we are looking in decoupling this functionality from AIR as it is today as it adds a lot of unnecessary complexity for 80 % of use cases which require simple mathematical rules. We are looking to improve on the function deployment experience so that users are able to generate their own KPI’s much easier and therefore be able to do monitoring on them. Finally in the meanwhile while we are moving towards alerting from charts on regular and calculated time series you will still be able to generate KPI’s using functions. It just wont be a tightly coupled experience as it is today.
How can SME create anomaly detection logic with output notifications? This is not a threshold alert like Arun is discussing now. Great question @Chris Selph! one of the things we are working towards next year is the ability to create monitoring jobs on charts calculations. This ability to do no code calculations should in essence allow our users to customize their monitoring models to solve more complex use cases.
Hei @Glen Accardo thanks again for reaching out regarding this and giving us valuable feedback. I will go ahead and add this to the backlog for us. I would also like to say that at the moment we are on a feature freeze for the powerBI connector due to the team being occupied with monitoring stabilisation within CDF but we should have a more open roadmap in the months to come.
Hei hei ! so sorry it took us a while to get back to you on this. We are working on a early adopter group for Direct Query at the moment and it should be alive and kicking soon so please stay tuned. In the meanwhile feel free to post any questions you have in this thread.
Could you also send us a link to the repository so we can take a look ? :)
@Hamdi Elhamdi @Bert Verstraete tagging for context.
This seems to be an issue with the poetry virtual environment being broken in githib actions and as a result it is not installing the correct packages. Is there a reason why this may have happened? did the poetry file get updated and the lock file was not updated to match the new poetry.yaml?
Hei hei thanks again for getting in touch with us . The team is looking into this issue as we speak.
Ok great looking into this and will get back to you with a response soon!
Hei hei could you clarify a bit more on what this is needed for ? so i can maybe suggest a potential solution ?
Hei hei this should be resolved with a simple poetry update click . This should update click to the latest version which should fix this issue.
Thanks for the great feedback on this. As you are aware AIR is going through a stability improvement phase and therefore the team is currently focused on that endaevour. This feature however is a great idea and will be added to the product backlog.
Hei hei so the SMS feature is an experimental feature and it is something that is still being tested. We would need to enable it in the backend by adding your project to the list of projects that this is enabled for.. I can do that if you can let me know which project this is for :) . Aa quick heads up though is that AIR is a restabilisation effort where we are moving all the components of AIR into CDF. In this effort we will be focusing on email notifications for now and SMS will be available in GA. It will however be added further down in the backlog as we release more features.
Yup i would create another model in AIR using python to do this with alerts being generated if the values go in a given range. Myself or a data scientist at Cognite can help with this!
Already have an account? Login
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.