Impact 2024: The Industrial Data and AI Conference for and by Users | Nominate Speakers Now for a Ch...
Hi @rmaidla thanks again for the great question and i apologize for the delayed response on my part. So every event that has been generated is written to Cognite Data Fusion as an alert event. So you could technically create alerts on past occurring events but not on current ones since they are still to be processed if that makes sense . But I would love for the use case to be clarified a bit more so I could potentially provide some solutions.
Great idea for a feature but this would just be a matter of creating a new model to reflect this. One of the key principles of AIR is to enable flexible models based on the use case. In this case you could have an input be a time frame and then you could check for the value being in a certain time frame and then creating alerts based on that.
Hei hei yes that would be possible. You would need to write a model that checks for a given time series value and creates alerts based on if this value remains below a threshold for a given time frame.
Hei hei great question and i apologise for the delayed response on my end regarding this. This is currently not possible in the AIR UI today but you could functionally do this by hardcoding the value in the backend. This is a great idea for a feature though and will add this to the backlog going forward.
Yup i would create another model in AIR using python to do this with alerts being generated if the values go in a given range. Myself or a data scientist at Cognite can help with this!
Hei hei so the SMS feature is an experimental feature and it is something that is still being tested. We would need to enable it in the backend by adding your project to the list of projects that this is enabled for.. I can do that if you can let me know which project this is for :) . Aa quick heads up though is that AIR is a restabilisation effort where we are moving all the components of AIR into CDF. In this effort we will be focusing on email notifications for now and SMS will be available in GA. It will however be added further down in the backlog as we release more features.
Thanks for the great feedback on this. As you are aware AIR is going through a stability improvement phase and therefore the team is currently focused on that endaevour. This feature however is a great idea and will be added to the product backlog.
Hei hei this should be resolved with a simple poetry update click . This should update click to the latest version which should fix this issue.
Hei hei could you clarify a bit more on what this is needed for ? so i can maybe suggest a potential solution ?
Ok great looking into this and will get back to you with a response soon!
Hei hei thanks again for getting in touch with us . The team is looking into this issue as we speak.
This seems to be an issue with the poetry virtual environment being broken in githib actions and as a result it is not installing the correct packages. Is there a reason why this may have happened? did the poetry file get updated and the lock file was not updated to match the new poetry.yaml?
@Hamdi Elhamdi @Bert Verstraete tagging for context.
Could you also send us a link to the repository so we can take a look ? :)
Hei hei ! so sorry it took us a while to get back to you on this. We are working on a early adopter group for Direct Query at the moment and it should be alive and kicking soon so please stay tuned. In the meanwhile feel free to post any questions you have in this thread.
Hei @Glen Accardo thanks again for reaching out regarding this and giving us valuable feedback. I will go ahead and add this to the backlog for us. I would also like to say that at the moment we are on a feature freeze for the powerBI connector due to the team being occupied with monitoring stabilisation within CDF but we should have a more open roadmap in the months to come.
How can SME create anomaly detection logic with output notifications? This is not a threshold alert like Arun is discussing now. Great question @Chris Selph! one of the things we are working towards next year is the ability to create monitoring jobs on charts calculations. This ability to do no code calculations should in essence allow our users to customize their monitoring models to solve more complex use cases.
@ibrahim.alsyed yes python will continue to be a part of our infrastructure but we are looking in decoupling this functionality from AIR as it is today as it adds a lot of unnecessary complexity for 80 % of use cases which require simple mathematical rules. We are looking to improve on the function deployment experience so that users are able to generate their own KPI’s much easier and therefore be able to do monitoring on them. Finally in the meanwhile while we are moving towards alerting from charts on regular and calculated time series you will still be able to generate KPI’s using functions. It just wont be a tightly coupled experience as it is today.
@Arun Arunachalam and @Knut Vidvei - The one thing we feel is missing from this list that would make alerting far more valuable to end users (and differentiating from other alerts we already have on our facilities) is the ability to generate an alert workflow given we rarely rely on a single alert to take action. Example: if we’re monitoring the flow through a pump and we want to be alerted when the pump is encountering issues performing, we expect the flow to go to zero when the entire facility is down. The only time this alert would be of value is if the facility is online and the pump rate drops below a specified threshold. Example 2 - most of our RCA’s involve some sort of series of events that are currently captured as RCA’s that are static rather than building it in to a monitoring system. By allowing for this workflow to be “automated” within AIR, we can now ensure that these events are being monitored to be prevented in the future and it would be beneficial to pull up as part
Is Alerting on possible using Time Series data? What if i want alerting on transaction such as Work Orders or Product Recipes that change etc? So alerting as we are working on today is focused on a time series approach and providing a stable and scalable experience not only across numbers but across use cases. Although alerting on events is not something that is coming soon, what we are working on is to enable the infrastructure to allow this to be possible in a scalable manner while we understand the use cases and workflows that into such a event driven system. We are hoping to reach out later in the year to our users as we move towards a more stable product to understand and solve this use cases better so stay tuned. I do wonder when you say work orders and product recipes what would a representation of that look like in CDF ?
Can we do unsupervised Machine Learning Anomaly Detection on Time Series Trends in AIR? This is a bit hard to answer without knowing the details of the models and the inputs that go into these models. With the alerting approach that we going with today the way you could achieve this is by writing a cognite function that runs a pretrained model and writes some kind of KPI. You could then potentially do alerting on these as we expand on more thresholding capabilities. Would be a cool exercise for us to explore these use cases together and figure out what he missing pieces are. I see strong potential in refining that cognite function experience as we get more user input and providing less friction between deploying these models and setting up alerting.
It would be events. Yeah if this is the case then yeah its something we will start looking into as we come upon a more stable experience for time series alerting in CDF both feature wise and performance wise in the months to come. While we are working on this we will reach out to better understand these use cases and the flows that go into such a event driven alerting system.
This is a fantastic use case for AIR but how would we do this if you decouple python modelling from AIR. Thanks again for the great question @ibrahim.alsyed . The way i would recommend working with this use case is to write a function in cognite functions that writes a KPI and then you could select the thresholds to be applied on the KPI based on your use case. One of goals in the future is to make this process much easier to do and reduce the time from KPI generation using user generated code to value.
Cognite Charts has a good features that helps in analyzing time series data. One suggestion related to the Threshold feature is to add the ability to trigger notifications based on threshold exceedance. The notification should be customizable to avoid too many notifications. In addition, it would be great also to provide more options for the types of notifications. Example, Email, SMS or popup. This feature will be very useful for, domain experts, operators and engineers to setup an offline monitoring task that can notify them to go back to CDF whenever a notification is triggered to analyse. I think it will increase Cognite Charts utilization by making it a great source of root cause analysis whenever a notification is triggered. Hei Adeep thanks again for your great insights on this. On a very positive note the team is currently working on this and you will be able to set monitoring on time series within charts on thresholds with an ability to receive email notifications. So stay t
Great feedback on this ! Would love to get more of an understanding but would this be related to having an endpoint way of subscribing to alerts ?
Already have an account? Login
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.