Skip to main content
Solved

2_List_Search_Retrieve.ipynb, np not found error


Forum|alt.badge.img+1
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[97], line 1
----> 1 c.time_series.data.retrieve_dataframe(id=239507005016003,
      2                     end='1000w-ago',
      3                     aggregates=["average","sum"],
      4                     granularity="1h")

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:975, in DatapointsAPI.retrieve_dataframe(self, id, external_id, start, end, aggregates, granularity, limit, include_outside_points, ignore_unknown_ids, uniform_index, include_aggregate_name, include_granularity_name, column_names)
    973 fetcher = select_dps_fetch_strategy(self, user_query=query)
    974 if not uniform_index:
--> 975     return fetcher.fetch_all_datapoints_numpy().to_pandas(
    976         column_names, include_aggregate_name, include_granularity_name
    977     )
    978 # Uniform index requires extra validation and processing:
    979 grans_given = {q.granularity for q in fetcher.all_queries}

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:175, in DpsFetchStrategy.fetch_all_datapoints_numpy(self)
    173 def fetch_all_datapoints_numpy(self) -> DatapointsArrayList:
    174     with get_priority_executor(max_workers=self.max_workers) as pool:
--> 175         ordered_results = self._fetch_all(pool, use_numpy=True)
    176     return self._finalize_tasks(ordered_results, resource_lst=DatapointsArrayList)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:228, in EagerDpsFetcher._fetch_all(self, pool, use_numpy)
    227 def _fetch_all(self, pool: PriorityThreadPoolExecutor, use_numpy: bool) -> List[BaseConcurrentTask]:
--> 228     futures_dct, ts_task_lookup = self._create_initial_tasks(pool, use_numpy)
    230     # Run until all top level tasks are complete:
    231     while futures_dct:

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:262, in EagerDpsFetcher._create_initial_tasks(self, pool, use_numpy)
    260 ts_task_lookup, payload = {}, {"ignoreUnknownIds": False}
    261 for query in self.all_queries:
--> 262     ts_task = ts_task_lookup[query] = query.ts_task_type(query=query, eager_mode=True, use_numpy=use_numpy)
    263     for subtask in ts_task.split_into_subtasks(self.max_workers, self.n_queries):
    264         future = pool.submit(self.__request_datapoints_jit, subtask, payload, priority=subtask.priority)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoint_tasks.py:1286, in ParallelUnlimitedAggTask.__init__(self, **kwargs)
   1285 def __init__(self, **kwargs: Any) -> None:
-> 1286     super().__init__(**kwargs)
   1287     # This entire method just to tell mypy:
   1288     assert isinstance(self.query, _SingleTSQueryAggUnlimited)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoint_tasks.py:1140, in BaseConcurrentAggTask.__init__(self, query, use_numpy, **kwargs)
   1138 def __init__(self, *, query: _SingleTSQueryAgg, use_numpy: bool, **kwargs: Any) -> None:
   1139     aggregates_cc = query.aggregates_cc
-> 1140     self._set_aggregate_vars(aggregates_cc, use_numpy)
   1141     super().__init__(query=query, use_numpy=use_numpy, **kwargs)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoint_tasks.py:1169, in BaseConcurrentAggTask._set_aggregate_vars(self, aggregates_cc, use_numpy)
   1167     self.dtype_aggs: np.dtype[Any] = np.dtype(np.float64)
   1168 else:  # (.., 1) is deprecated for some reason
-> 1169     self.dtype_aggs = np.dtype((np.float64, len(self.float_aggs)))

NameError: name 'np' is not defined

The packages in virtual environment.

 

Best answer by Håkon V. Treider

Mysterious. Is it possible that you imported the CogniteClient before numpy was installed? If you restart the notebook are you able to reproduce the error?

View original
Did this topic help you find an answer to your question?

Forum|alt.badge.img

Thanks for reporting this! Could you double-check your virtual environment packages? This looks like a bug that was fixed in 5.7.3

What is the output of running:

from numpy.version import version
print(version)
from cognite.client import __version__
print(__version__)

 


Forum|alt.badge.img+1

I am using latest version of the Cognite sdk I guess.

 If you have any workaround, please let me know.


Forum|alt.badge.img

Hmm… this seems like an impossible bug (obviously not though 😅). 
 

The error originates after a call to retrieve_dataframe, which does a test import for both numpy and pandas to make sure both are available. This test passes for some reason 🤔 However, a previous test import of numpy has failed, leading to the SDK thinking it has been installed without numpy support. I don’t see from the code how this is possible.

Could you try to print the following:

from cognite.client.data_classes.datapoints import NUMPY_IS_AVAILABLE


Forum|alt.badge.img+1

It gives false as it should be to occur the error.
 

In the github repo.
 

I think it is importing from wrong folder. (Not sure though). This may work.
 

 


Forum|alt.badge.img

Mysterious. Is it possible that you imported the CogniteClient before numpy was installed? If you restart the notebook are you able to reproduce the error?


Forum|alt.badge.img+1

I restarted the environment and tried, and it executed without an error.

Thank You for the guidance.


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings