Topics started by Niranjan Madhukar Karvekar

7 Topics

N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

Flexible data modeling - transformation to JsonObject type

Hi, I have created one FDM model having JsonObject field. In the transformation query I referred that field , example query:SELECT Series_Title as externalId,       Series_Title as name,       Overview as description,       int(Gross) as gross,       float(IMDB_Rating) as imdbRating,       int(Runtime) as runTime,       int(Released_Year) as releasedYear,       '{"key1":"value1","key2":{"key3":12,"key4":"hello","key5":{"key6":10}}}' as dataFROM movies.moviesFDM model is:type Movie {  name: String!  description: String  watchedIt: Boolean  imdbRating: Float  releasedYear: Int  runTime: Int  gross: Int  actors: [Actor]  data: JSONObject}  The transformation is failing with this error:Unknown property type Json. May be I am missing something, Can you please help?

10
Dilini Fernando
Seasoned Practitioner
1 year ago
N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

transformations-cli with FDM

Hi,I am trying to run transformations-cli locally with below command:transformations-cli deploy . (The current directory has manifest.yaml and transformation.sql files) I am getting below error:Deploying transformations...Failed to parse transformation config, please check that you conform required fields and format: Invalid config: can not match type "dict" to any type of "destination" union: typing.Union[cognite.transformations_cli.commands.deploy.transformation_types.DestinationType, cognite.transformations_cli.commands.deploy.transformation_types.DestinationConfig, cognite.transformations_cli.commands.deploy.transformation_types.RawDestinationConfig, cognite.transformations_cli.commands.deploy.transformation_types.SequenceRowsDestinationConfig, cognite.transformations_cli.commands.deploy.transformation_types.AlphaDMIDestinationConfig] Here is destination section of manifest file:destination:  viewSpaceExternalId: my-model-space-id  viewExternalId: CDFTimeSeries  viewVersion: 0_2  i

17
S
Seasoned
1 year ago
N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

FDM - Get Latest value

Hi,Let’s I have below data model created in FDM and there are 2 types Device and Sensor. Device is referring to other type Sensor.  Device  - name:String  - sensors:[Sensor]  Sensor  - name:String   - timestamp: Timestamp   - value: Float I have populated the data through transformation in this model. I wanted to check on how to get the latest timestamp and value for particular device and sensor through FDM GraphQL query.Example:Device : d1Sensor:name: s1timestamp:   2023-01-01 01:00value: 10.0 name: s1timestamp:   2023-01-02 01:00value: 20.0 name: s1timestamp:   2023-01-03 01:00value: 30.0 I am expecting to latest timestamp as 2023-01-03 01:00 and value as 30.0 

9
Anders Hafreager
Practitioner
1 year ago
N
Committed
Niranjan Madhukar KarvekarCommitted

How to use is_new() function with lastUpdatedTime from FDM type in transformationPlanned for development

Hi, When the data is in CDF RAW then we can use is_new function with lastUpdatedTime  in transformation and this works fine but how can we use that function for data coming from FDM model. Let’s say, from CDF RAW we are populating the data in FDM domain model and we are considering only incremental data with is_new function in transformations.This is the domain model (space: dm):type Entity @view(space: "dm", version: "1"){name:String} This is transformation query to populate above Entity type:select  distinct concat('wellbore:',WellUWI)  as externalId,  WellName as name, FROM  `db1`.`wellbores`where is_new(        "db1:wellbores",        lastUpdatedTime      )This is good and we can see only incremental data from CDF RAW in the domain model.  Now this is the solution model (sol_model):type WELLBORE{name:String }This is transformation query to populate above WELLBORE type ( I can directly map the name of WELLBORE of solution model to ENTITY of domain model but for this example I am ign

10
N
Committed
4 months ago
N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

DB extractor - storing incremental state for Timeseries query

Hi, I am observing below error(in bold) in extractor log for timeseries query: 2023-01-24 03:31:03.345 UTC [INFO    ] QueryExecutor_0 - Starting extraction of avocet-timeseries2023-01-24 03:31:03.345 UTC [DEBUG   ] QueryExecutor_0 - Query: 'SELECT ITEM_ID + PROPERTY_TYPE+'Avocet' as "externalId", START_DATETIME as timestamp, PROPERTY_VALUE as value FROM ITEM_PROPERTY1 WHERE START_DATETIME is not null and PROPERTY_VALUE is not null and LAST_UPDT_DATE is not null ORDER BY LAST_UPDT_DATE ASC'2023-01-24 03:31:03.612 UTC [INFO    ] QueryExecutor_0 - No more rows for avocet-timeseries. 2 rows extracted in 0.267 seconds2023-01-24 03:31:03.873 UTC [DEBUG   ] ThreadPoolExecutor-9_0 - https://westeurope-1.cognitedata.com:443 "POST /api/v1/projects/slb-pdf/timeseries/data HTTP/1.1" 200 222023-01-24 03:31:03.875 UTC [DEBUG   ] ThreadPoolExecutor-9_0 - HTTP/1.1 POST https://westeurope-1.cognitedata.com/api/v1/projects/slb-pdf/timeseries/data 2002023-01-24 03:31:03.877 UTC [INFO    ] QueryExecutor_0

7
N
Committed
10 months ago
N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

Transformation to RAW

Hi,I have data in raw table in this format:Table1:id p1 p2 p3 1 2.0 3.0 4.0 2 10.0 15.0 20.0  I wanted to transform this details into following format through transformation id property value 1 p1 2.0 1 p2 3.0 1 p3 4.0 2 p1 10.0 2 p2 15.0 2 p3 20.0   so I have written following transformation query:SELECT id, property, valueFROM `db1`.`Table1`UNPIVOT (value FOR property IN (p1, p2,p3)) UP This query works in SQL studio, but in COGNITE transformation UI , I see below error:Mismatched input 'FOR' expecting {')', ',', '-'}(line 3, pos 15) Can you please help to understand what is missing in this?  

2
N
Committed
1 year ago
N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

DB Extractor - Too many concurrent requests in pod

Hi, I see this error in extractor log, I hope even if this happens , it will be retried and there will not be the data loss. Please correct if I am wrong or if there are ways to change any configuration in extractor to avoid this.  2023-03-07 05:42:04.233 UTC [WARNING ] QueryExecutor_3 - Too many concurrent requests in pod | code: 503

1
M
Practitioner
1 year ago

Badge winners

Show all badges
Terms & ConditionsCookie Settings