Seasoned ⭐️⭐️

Solved

I would like to write unittest for Extractor run main function, Please help me to find mistakes in my code.

Forum|Forum|2 years ago
November 5, 2023
6 replies
174 views

Hi Team,

Please help me to find the mistakes in my code.

Let me showcase the dummy code
main code:

def main() -> None:
"""
Main entrypoint
"""
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
config_values_vault.set_vara()
with Extractor(
name="SAP_Extractor",
description="An extractor to extract asset hierarchy from SAP based on root node",
config_class=SapConfig,
# version=__version__,
# debugger
version="1.0.0",
run_handle=run,
metrics=metrics,
config_file_path= os.path.join(BASE_DIR, 'config.yaml'),

) as extractor:
extractor.run()

I have build code unittest for above code:

def test_main():
with patch('os.path') as path_mock:
with patch.object(path_mock, 'abspath') as mock_abspath:
with patch.object(path_mock, 'dirname') as mock_dirname:
with patch.object(path_mock,'join') as mock_join:
mock_abspath.return_value = '/path/to/yaml_file.yaml'
mock_dirname.side_effect = lambda x: x
with patch('sap_extractor.config_values_vault.set_vara'):
with patch('cognite.extractorutils.Extractor.run') as extractor_mock:
with patch('sap_extractor.config.SapConfig'):
with patch('sap_extractor.metrics'):
with patch('builtins.open',create=True) as mock_open:
mock_file = Mock()
mock_file.read.return_value = 'abc'
mock_open.return_value.__enter__.return_value = mock_file
with patch('sap_extractor.extractor.run') as run_mock:
run_mock.return_value = Mock()
extractor.main()

When I try to execute it got stuck not going forward and neither it’s fail or pass. I feel something I miss or did mistake in mocking the objects. Please help me to resolve it.

Thanks in Advance
Harshita

Best answer by Nimesha

Hi Harshita,

In the provided test, you’ve mocked the following paths;

mock_abspath.return_value = '/path/to/yaml_file.yaml'
mock_dirname.side_effect = lambda x: x

Ensure that the mock paths match the expected path in the main function. If your BASE_DIR is not '/path/to/', you should adjust the mocked paths accordingly.

Example;

mock_abspath.return_value = '/path/to/'
mock_dirname.side_effect = lambda x: '/path/to/'
mock_join.side_effect = lambda *args: '/path/to/yaml_file.yaml'

If the issue persists, you might want to add print statements or use a debugger to understand where the code is getting stuck.

For example, print statements like this;

from unittest.mock import patch, Mock
from sap_extractor import extractor

@patch('sap_extractor.config_values_vault.set_vara')
@patch('cognite.extractorutils.Extractor.run')
@patch('sap_extractor.config.SapConfig')
@patch('sap_extractor.metrics')
@patch('builtins.open', create=True)
@patch('os.path.abspath')
@patch('os.path.dirname')
@patch('os.path.join')
def test_main(
    mock_join, mock_dirname, mock_abspath, mock_open,
    mock_metrics, mock_sap_config, mock_extractor_run, mock_set_vara
):
    mock_abspath.return_value = '/path/to/yaml_file.yaml'
    mock_dirname.side_effect = lambda x: x
    mock_open.return_value.__enter__.return_value.read.return_value = 'abc'

    print("Before main")

    with patch.object(extractor, 'run') as mock_run:
        mock_run.return_value = Mock()

        print("Before main call")
        extractor.main()

        print("After main call")

    assert mock_set_vara.called
    assert mock_sap_config.called
    assert mock_metrics.called
    assert mock_extractor_run.called
    assert mock_open.called
    assert mock_run.called

N

Nimesha

Cognite

Hi Harshita,

Thank you for your question.

It seems like your test is stuck due to a deadlock or infinite loop in your code or the test itself. Could you please simplify the test structure as show below?

from unittest.mock import patch, Mock
from sap_extractor import extractor

@patch('sap_extractor.config_values_vault.set_vara')
@patch('cognite.extractorutils.Extractor.run')
@patch('sap_extractor.config.SapConfig')
@patch('sap_extractor.metrics')
@patch('builtins.open', create=True)
@patch('os.path.abspath')
@patch('os.path.dirname')
@patch('os.path.join')
def test_main(
    mock_join, mock_dirname, mock_abspath, mock_open,
    mock_metrics, mock_sap_config, mock_extractor_run, mock_set_vara
):
    mock_abspath.return_value = '/path/to/yaml_file.yaml'
    mock_dirname.side_effect = lambda x: x
    mock_open.return_value.__enter__.return_value.read.return_value = 'abc'

    with patch.object(extractor, 'run') as mock_run:
        mock_run.return_value = Mock()

        extractor.main()

    assert mock_set_vara.called
    assert mock_sap_config.called
    assert mock_metrics.called
    assert mock_extractor_run.called
    assert mock_open.called
    assert mock_run.called

If the issue persists, please feel free to get back.

H

Harshita SaiAuthor

Seasoned ⭐️⭐️

Hi Nimesha,

Yeah I have simplified.

still it’s stuck in between. May be we need to mock the yaml file properly?

os.path.join(BASE_DIR, 'config.yaml'),

I am in confuse like I did not mock the above line properly? Can you help me how to mock the yaml file if you feel the same.

Even if you feel something else is going wrong also please feel free to point out.

Thanks,

Sai Harshita

N

NimeshaAnswer

Cognite

Hi Harshita,

In the provided test, you’ve mocked the following paths;

mock_abspath.return_value = '/path/to/yaml_file.yaml'
mock_dirname.side_effect = lambda x: x

Ensure that the mock paths match the expected path in the main function. If your BASE_DIR is not '/path/to/', you should adjust the mocked paths accordingly.

Example;

mock_abspath.return_value = '/path/to/'
mock_dirname.side_effect = lambda x: '/path/to/'
mock_join.side_effect = lambda *args: '/path/to/yaml_file.yaml'

If the issue persists, you might want to add print statements or use a debugger to understand where the code is getting stuck.

For example, print statements like this;

from unittest.mock import patch, Mock
from sap_extractor import extractor

@patch('sap_extractor.config_values_vault.set_vara')
@patch('cognite.extractorutils.Extractor.run')
@patch('sap_extractor.config.SapConfig')
@patch('sap_extractor.metrics')
@patch('builtins.open', create=True)
@patch('os.path.abspath')
@patch('os.path.dirname')
@patch('os.path.join')
def test_main(
    mock_join, mock_dirname, mock_abspath, mock_open,
    mock_metrics, mock_sap_config, mock_extractor_run, mock_set_vara
):
    mock_abspath.return_value = '/path/to/yaml_file.yaml'
    mock_dirname.side_effect = lambda x: x
    mock_open.return_value.__enter__.return_value.read.return_value = 'abc'

    print("Before main")

    with patch.object(extractor, 'run') as mock_run:
        mock_run.return_value = Mock()

        print("Before main call")
        extractor.main()

        print("After main call")

    assert mock_set_vara.called
    assert mock_sap_config.called
    assert mock_metrics.called
    assert mock_extractor_run.called
    assert mock_open.called
    assert mock_run.called

H

Harshita SaiAuthor

Seasoned ⭐️⭐️

Yeah I have tried by debugging after calling main function it’s not coming back.

from cognite.extractorutils import Extractor
def main() -> None:
"""
Main entrypoint
"""
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
config_values_vault.set_vara()
with Extractor(
name="SAP_Extractor",
description="An extractor to extract asset hierarchy from SAP based on root node",
config_class=SapConfig,
# version=__version__,
# debugger
version="1.0.0",
run_handle=run,
metrics=metrics,
config_file_path= os.path.join(BASE_DIR, 'config.yaml'),
) as extractor:
extractor.run()

The above is original code.

Anita Hæhre

Head of Community

Hi Community! if there is someone here who can provide assistance on the above from @Harshita Sai we would greatly appreciate it. Your collective knowledge and experience make this community a valuable resource, so if you know how to help out here, we’d love to hear from you 🏅

Anita Hæhre

Isha Thapliyal

Expert ⭐️⭐️⭐️⭐️

Hi @Harshita Sai

Wanted to check in with you regarding the post you made a while back. We haven’t seen any updates, so I’m reaching out to see if everything has been resolved or if you still need assistance.

Looking forward to hearing from you!

Thanks

Isha Thapliyal

Sign up

Welcome to Cognite Hub