Solved

Facing issue while writing test cases using monkey_cognite_patch.

V
Rank
Badge +4

Hi Team, 

We are working on a POC , where we need to create unit test cases where we are facing different issues .For reference we are using code from Cognite Hub

Below attaching our code and output expectations .
Please help with the approach to resolve this issue 

ts_retrieved = [

    TimeSeries(

    id= 3973245479016710,

    external_id= "Brazil:3W:Well-Bore-19B_target_throughput",

    name= "Brazil:3W:Well-Bore-19B_target_throughput",

    is_string= False,

    metadata= {},

    unit= "Barrel",

    asset_id= 3332326574821917,

    is_step= False,

    description= "target volume for a oil well bore",

    security_categories= [],

    data_set_id= 8739280581150161,

    created_time= "2023-05-02 09:37:04.944000",

    last_updated_time= "2023-05-02 09:37:04.944000"

)

]

@pytest.fixture

def cognite_client_mock():

    with monkeypatch_cognite_client() as client:

          client.assets.retrieve().time_series().get().return_value = ts_retrieved

        return client  

 

Method code 
 

        ts_data = c.assets.retrieve(external_id=asset_name).time_series().get(external_id=f'{asset_name}_{ts_type}').to_pandas()

        ts_internal_id = int(ts_data.loc["id"])

        print(f'ts_internal_id {ts_internal_id}')

        print(f'ts_internal_id {ts_data.head(5)}')

Expectation from print is ts_internal_id = 3973245479016710
Actual Output - 


Please provide suggestions on how to achieve the desired output in unit testing the cognite calls.
Thanks.​​​​​​​

 

icon

Best answer by erlend.vollset 6 July 2023, 14:31

View original

10 replies

H
Rank
Userlevel 4
Badge

You chain a lot of methods here and so mocking will be a bit entangled as well. First off, you should not call the methods on the mock when setting a return value:

Wrong: client.assets.retrieve().return_value = asset

Correct: client.assets.retrieve.return_value = asset

 

Secondly, you need to think about what is returned by these intermediate chained calls, i.e. you first get an asset object, then you ask for the time series connected to it. This gives you a TimeSeriesList object. On this object, you get the specific time series by using the .get method supplying an identifier etc, etc, etc.

You should probably mock these objects separately, i.e.

mock_ts_list = ...
mock_asset = ...
mock_asset.time_series.return_value = mock_ts_list

client.assets.retrieve.return_value = mock_asset

Hope this have given you a few pointers to mocking! Let me know if you have further follow-up questions.

S
Rank
Badge

I tried the same process like you mentioned above and facing the below issue. Could you please tell more about it. Screenshot provided below.

Error : AttributeError: 'method' object has no attribute 'return_value'

Method Code : 

ts_data = c.assets.retrieve(external_id=asset_name).time_series().get(external_id=f'{asset_name}_{ts_type}').to_pandas()
print(f'ts_internal_id {ts_data}')

conftest file :

mock_ts_list = [
    TimeSeries(
        id= 11400444674081,
        external_id= "Brazil:3W:Well-Bore-19B_estimated_throughput",
        name= "Brazil:3W:Well-Bore-19B_estimated_throughput",
        is_string= False,
        metadata= {},
        unit= "Barrel",
        asset_id= 33342657421917,
        is_step= False,
        description= "target volume for a oil well bore",
        security_categories= [],
        data_set_id= 873920581150161,
        created_time= "2023-05-02 09:37:04.944000",
        last_updated_time= "2023-05-02 09:37:04.944000"
    )]

mock_asset=Asset(
           id=397444479016710,
           external_id="Brazil:3W:Well-Bore-19B_target_throughput",
           name="Brazil:3W:Well-Bore-19B_target_throughput",
           metadata={
               "some_number": str(9999999),
               "some_number_times_2": str(99999999999)
           },

       )

@pytest.fixture
def cognite_client_mock():
    mock_asset_name = "Brazil:3W:Well-Bore-19B"
    mock_allowance_unscheduled_events = 0.10
    mock_data_set_id= 12345678987643
    mock_ts_type= "estimated_throughput"
    with monkeypatch_cognite_client() as client:
        mock_asset.time_series.return_value = mock_ts_list
        client.assets.retrieve.return_value=mock_asset
        return client

 

 

H
Rank
Userlevel 4
Badge

Your mock_asset, is not a mock. You could just use MagicMock directly, or better, spec the Asset data class:

from unittest.mock import create_autospec
from cognite.client.data_classes import Asset

mock_asset = create_autospec(Asset, spec_set=True)

 

V
Rank
Badge +4

Hi @Håkon V. Treider could you give detailed example using above code.

V
Rank
Badge +4

Hi @Håkon V. Treider We tried using monkeypatch too , in that we are getting stuck at one line of code which is giving below error 

TypeError: '<=' not supported between instances of 'int' and 'MagicMock'

 

Code we are using is 

        df_filtered_timefilter = df_filtered[(event_start_date <= df_filtered["start_time"]) & (df_filtered["end_time"] <= event_end_date)]

In this event_start_date is mocked but df_filtered is coming as 

<MagicMock name='mock.assets.retrieve().subtree().events().to_pandas().__getitem__()' id='2821478081008'>

Any solution you could recomment for this ?

E
Rank

Second what @Håkon V. Treider writes here. You need to consider intermediate chained calls. This works:

from unittest.mock import MagicMock

from cognite.client.data_classes import TimeSeries, TimeSeriesList, Asset
from cognite.client.testing import monkeypatch_cognite_client

ts_retrieved = [
    TimeSeries(
        id=3973245479016710,
        external_id="Brazil:3W:Well-Bore-19B_target_throughput",
        name="Brazil:3W:Well-Bore-19B_target_throughput",
        is_string=False,
        metadata={},
        unit="Barrel",
        asset_id=3332326574821917,
        is_step=False,
        description="target volume for a oil well bore",
        security_categories=[],
        data_set_id=8739280581150161,
        created_time=123,
        last_updated_time=345
    )

]

asset_name = "some_asset"

with monkeypatch_cognite_client() as client:
    asset_retrieve_response_mock = MagicMock()
    asset_retrieve_response_mock.time_series.return_value = TimeSeriesList(ts_retrieved)
    client.assets.retrieve.return_value = asset_retrieve_response_mock

    ts_data = client.assets.retrieve(external_id=asset_name).time_series().get(external_id=ts_retrieved[0].external_id).to_pandas()

    ts_internal_id = int(ts_data.loc["id"])

    print(f'ts_internal_id {ts_internal_id}')
    print(f'ts_internal_id {ts_data.head(5)}')

 

V
Rank
Badge +4

Hi @erlend.vollset 
I have tried using monkey patch in your suggested pattern for example 
 

 

ts_retrieved1 = { "index":["id","external_id","name","is_string","metadata","unit","asset_id","is_step", "description","security_categories","data_set_id","created_time","last_updated_time"],

                 "value":[3973245479016710,"Brazil:3W:Well-Bore-19B_target_throughput","Brazil:3W:Well-Bore-19B_target_throughput",False,{}, "Barrel",3332326574821917,False,"target volume for a oil well bore",[],8739280581150161,"2023-05-02 09:37:04.944000", "2023-05-02 09:37:04.944000"]}

 

desired_value = pd.DataFrame(ts_retrieved1)

mock_to_pandas = MagicMock(return_value=desired_value)

mock_get= MagicMock()

mock_get.to_pandas = mock_to_pandas

mock_timeseries = MagicMock()

mock_timeseries.get = MagicMock(return_value=mock_get)

mock_retrieve = MagicMock()

mock_retrieve.timeseries = MagicMock(return_value=mock_timeseries)
 

mock_client = MagicMock()

mock_client.assets.retrieve = MagicMock(return_value=mock_retrieve)

 


Like this i had to make 5 calls in a method .

Use case is in between these calls 
I have used is        
df_filtered_timefilter = df_filtered[(event_start_date <= df_filtered["start_time"]) & (df_filtered["end_time"] <= event_end_date)]

But i am getting this error 
<MagicMock name='mock.assets.retrieve().subtree().events().to_pandas().__getitem__()' id='2821478081008'>

and code is breaking is between these patches 
 

V
Rank
Badge +4

I can chain these intermediate calls as you suggested but my question is will this result a proper result which is expected with above like of code ?
Will it be able to pass that step ?
df_filtered_timefilter = df_filtered[(event_start_date <= df_filtered["start_time"]) & (df_filtered["end_time"] <= event_end_date)]
Or will these patches will be considered as real df so that the errored line could pass through to next step.

H
Rank
Userlevel 4
Badge

In @erlend.vollset code above, ts_data is an actual pandas DataFrame, which allows later pandas “data wrangling” code to succeed (in the test). I suspect one of your variables named “df_*” are still a mock.

(Note that Erlend sets the return value to a TimeSeriesList. The later .get call then uses the actual get-method on the object (i.e. not some mocked method). Same goes with the .to_pandas call after that.)

C
Rank
Userlevel 3

Hi @vidhi.c.jain , did the above help?

Reply


Terms & ConditionsCookie Settings