Skip to main content
Answer

2_List_Search_Retrieve.ipynb, np not found error

  • June 27, 2023
  • 6 replies
  • 79 views

Forum|alt.badge.img+1
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[97], line 1
----> 1 c.time_series.data.retrieve_dataframe(id=239507005016003,
2 end='1000w-ago',
3 aggregates=["average","sum"],
4 granularity="1h")

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:975, in DatapointsAPI.retrieve_dataframe(self, id, external_id, start, end, aggregates, granularity, limit, include_outside_points, ignore_unknown_ids, uniform_index, include_aggregate_name, include_granularity_name, column_names)
973 fetcher = select_dps_fetch_strategy(self, user_query=query)
974 if not uniform_index:
--> 975 return fetcher.fetch_all_datapoints_numpy().to_pandas(
976 column_names, include_aggregate_name, include_granularity_name
977 )
978 # Uniform index requires extra validation and processing:
979 grans_given = {q.granularity for q in fetcher.all_queries}

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:175, in DpsFetchStrategy.fetch_all_datapoints_numpy(self)
173 def fetch_all_datapoints_numpy(self) -> DatapointsArrayList:
174 with get_priority_executor(max_workers=self.max_workers) as pool:
--> 175 ordered_results = self._fetch_all(pool, use_numpy=True)
176 return self._finalize_tasks(ordered_results, resource_lst=DatapointsArrayList)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:228, in EagerDpsFetcher._fetch_all(self, pool, use_numpy)
227 def _fetch_all(self, pool: PriorityThreadPoolExecutor, use_numpy: bool) -> List[BaseConcurrentTask]:
--> 228 futures_dct, ts_task_lookup = self._create_initial_tasks(pool, use_numpy)
230 # Run until all top level tasks are complete:
231 while futures_dct:

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoints.py:262, in EagerDpsFetcher._create_initial_tasks(self, pool, use_numpy)
260 ts_task_lookup, payload = {}, {"ignoreUnknownIds": False}
261 for query in self.all_queries:
--> 262 ts_task = ts_task_lookup[query] = query.ts_task_type(query=query, eager_mode=True, use_numpy=use_numpy)
263 for subtask in ts_task.split_into_subtasks(self.max_workers, self.n_queries):
264 future = pool.submit(self.__request_datapoints_jit, subtask, payload, priority=subtask.priority)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoint_tasks.py:1286, in ParallelUnlimitedAggTask.__init__(self, **kwargs)
1285 def __init__(self, **kwargs: Any) -> None:
-> 1286 super().__init__(**kwargs)
1287 # This entire method just to tell mypy:
1288 assert isinstance(self.query, _SingleTSQueryAggUnlimited)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoint_tasks.py:1140, in BaseConcurrentAggTask.__init__(self, query, use_numpy, **kwargs)
1138 def __init__(self, *, query: _SingleTSQueryAgg, use_numpy: bool, **kwargs: Any) -> None:
1139 aggregates_cc = query.aggregates_cc
-> 1140 self._set_aggregate_vars(aggregates_cc, use_numpy)
1141 super().__init__(query=query, use_numpy=use_numpy, **kwargs)

File c:\Users\Sudil\OneDrive - Creative Technology Solutions (Pvt) Ltd\Desktop\DataEngBasics\using-cognite-python-sdk\.venv\lib\site-packages\cognite\client\_api\datapoint_tasks.py:1169, in BaseConcurrentAggTask._set_aggregate_vars(self, aggregates_cc, use_numpy)
1167 self.dtype_aggs: np.dtype[Any] = np.dtype(np.float64)
1168 else: # (.., 1) is deprecated for some reason
-> 1169 self.dtype_aggs = np.dtype((np.float64, len(self.float_aggs)))

NameError: name 'np' is not defined

The packages in virtual environment.

 

Best answer by Håkon V. Treider

Mysterious. Is it possible that you imported the CogniteClient before numpy was installed? If you restart the notebook are you able to reproduce the error?

6 replies

Forum|alt.badge.img

Thanks for reporting this! Could you double-check your virtual environment packages? This looks like a bug that was fixed in 5.7.3

What is the output of running:

from numpy.version import version
print(version)
from cognite.client import __version__
print(__version__)

 


Forum|alt.badge.img+1

I am using latest version of the Cognite sdk I guess.

 If you have any workaround, please let me know.


Forum|alt.badge.img

Hmm… this seems like an impossible bug (obviously not though 😅). 
 

The error originates after a call to retrieve_dataframe, which does a test import for both numpy and pandas to make sure both are available. This test passes for some reason 🤔 However, a previous test import of numpy has failed, leading to the SDK thinking it has been installed without numpy support. I don’t see from the code how this is possible.

Could you try to print the following:

from cognite.client.data_classes.datapoints import NUMPY_IS_AVAILABLE


Forum|alt.badge.img+1

It gives false as it should be to occur the error.
 

In the github repo.
 

I think it is importing from wrong folder. (Not sure though). This may work.
 

 


Forum|alt.badge.img

Mysterious. Is it possible that you imported the CogniteClient before numpy was installed? If you restart the notebook are you able to reproduce the error?


Forum|alt.badge.img+1

I restarted the environment and tried, and it executed without an error.

Thank You for the guidance.