Skip to main content
Solved

I would like to write unittest for Extractor run main function, Please help me to find mistakes in my code.


Forum|alt.badge.img+6

Hi Team,

Please help me to find the mistakes in my code. 

Let me showcase the dummy code
main code:

def main() -> None:

    """

    Main entrypoint

    """

    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

    config_values_vault.set_vara()

    with Extractor(

        name="SAP_Extractor",

        description="An extractor to extract asset hierarchy from SAP based on root node",

        config_class=SapConfig,

        # version=__version__,

        # debugger

        version="1.0.0",

        run_handle=run,

        metrics=metrics,

        config_file_path= os.path.join(BASE_DIR, 'config.yaml'),

    ) as extractor:

        extractor.run()

 

I have build code unittest for above code:

 

def test_main():

    with patch('os.path') as path_mock:

        with patch.object(path_mock, 'abspath') as mock_abspath:

            with patch.object(path_mock, 'dirname') as mock_dirname:

                with patch.object(path_mock,'join') as mock_join:

                    mock_abspath.return_value = '/path/to/yaml_file.yaml'            

                    mock_dirname.side_effect = lambda x: x  

                    with patch('sap_extractor.config_values_vault.set_vara'):

                        with patch('cognite.extractorutils.Extractor.run') as extractor_mock:

                            with patch('sap_extractor.config.SapConfig'):

                                with patch('sap_extractor.metrics'):

                                    with patch('builtins.open',create=True) as mock_open:

                                        mock_file = Mock()

                                        mock_file.read.return_value = 'abc'

                                        mock_open.return_value.__enter__.return_value = mock_file

                                        with patch('sap_extractor.extractor.run') as run_mock:

                                            run_mock.return_value = Mock()

                                            extractor.main()  

When I try to execute it got stuck not going forward and neither it’s fail or pass. I feel something I miss or did mistake in mocking the objects. Please help me to resolve it.

Thanks in Advance
Harshita

Best answer by Nimesha

Hi Harshita,

In the provided test, you’ve mocked the following paths;

mock_abspath.return_value = '/path/to/yaml_file.yaml'
mock_dirname.side_effect = lambda x: x

Ensure that the mock paths match the expected path in the main function. If your BASE_DIR is not '/path/to/', you should adjust the mocked paths accordingly.

Example;

mock_abspath.return_value = '/path/to/'
mock_dirname.side_effect = lambda x: '/path/to/'
mock_join.side_effect = lambda *args: '/path/to/yaml_file.yaml'

If the issue persists, you might want to add print statements or use a debugger to understand where the code is getting stuck.

For example, print statements like this;

from unittest.mock import patch, Mock
from sap_extractor import extractor

@patch('sap_extractor.config_values_vault.set_vara')
@patch('cognite.extractorutils.Extractor.run')
@patch('sap_extractor.config.SapConfig')
@patch('sap_extractor.metrics')
@patch('builtins.open', create=True)
@patch('os.path.abspath')
@patch('os.path.dirname')
@patch('os.path.join')
def test_main(
    mock_join, mock_dirname, mock_abspath, mock_open,
    mock_metrics, mock_sap_config, mock_extractor_run, mock_set_vara
):
    mock_abspath.return_value = '/path/to/yaml_file.yaml'
    mock_dirname.side_effect = lambda x: x
    mock_open.return_value.__enter__.return_value.read.return_value = 'abc'

    print("Before main")

    with patch.object(extractor, 'run') as mock_run:
        mock_run.return_value = Mock()

        print("Before main call")
        extractor.main()

        print("After main call")

    assert mock_set_vara.called
    assert mock_sap_config.called
    assert mock_metrics.called
    assert mock_extractor_run.called
    assert mock_open.called
    assert mock_run.called

 

 

 

View original
Did this topic help you find an answer to your question?

  • Cognite
  • November 6, 2023

Hi Harshita,

Thank you for your question.

It seems like your test is stuck due to a deadlock or infinite loop in your code or the test itself. Could you please simplify the test structure as show below?

from unittest.mock import patch, Mock
from sap_extractor import extractor

@patch('sap_extractor.config_values_vault.set_vara')
@patch('cognite.extractorutils.Extractor.run')
@patch('sap_extractor.config.SapConfig')
@patch('sap_extractor.metrics')
@patch('builtins.open', create=True)
@patch('os.path.abspath')
@patch('os.path.dirname')
@patch('os.path.join')
def test_main(
    mock_join, mock_dirname, mock_abspath, mock_open,
    mock_metrics, mock_sap_config, mock_extractor_run, mock_set_vara
):
    mock_abspath.return_value = '/path/to/yaml_file.yaml'
    mock_dirname.side_effect = lambda x: x
    mock_open.return_value.__enter__.return_value.read.return_value = 'abc'

    with patch.object(extractor, 'run') as mock_run:
        mock_run.return_value = Mock()

        extractor.main()

    assert mock_set_vara.called
    assert mock_sap_config.called
    assert mock_metrics.called
    assert mock_extractor_run.called
    assert mock_open.called
    assert mock_run.called

If the issue persists, please feel free to get back. 

 


Forum|alt.badge.img+6

Hi Nimesha,

Yeah I have simplified.

still it’s stuck in between. May be we need to mock the yaml file properly?

os.path.join(BASE_DIR, 'config.yaml'),

I am in confuse like I did not mock the above line properly? Can you help me how to mock the yaml file if you feel the same.

Even if you feel something else is going wrong also please feel free to point out.

Thanks,

Sai Harshita


  • Cognite
  • November 6, 2023

Hi Harshita,

In the provided test, you’ve mocked the following paths;

mock_abspath.return_value = '/path/to/yaml_file.yaml'
mock_dirname.side_effect = lambda x: x

Ensure that the mock paths match the expected path in the main function. If your BASE_DIR is not '/path/to/', you should adjust the mocked paths accordingly.

Example;

mock_abspath.return_value = '/path/to/'
mock_dirname.side_effect = lambda x: '/path/to/'
mock_join.side_effect = lambda *args: '/path/to/yaml_file.yaml'

If the issue persists, you might want to add print statements or use a debugger to understand where the code is getting stuck.

For example, print statements like this;

from unittest.mock import patch, Mock
from sap_extractor import extractor

@patch('sap_extractor.config_values_vault.set_vara')
@patch('cognite.extractorutils.Extractor.run')
@patch('sap_extractor.config.SapConfig')
@patch('sap_extractor.metrics')
@patch('builtins.open', create=True)
@patch('os.path.abspath')
@patch('os.path.dirname')
@patch('os.path.join')
def test_main(
    mock_join, mock_dirname, mock_abspath, mock_open,
    mock_metrics, mock_sap_config, mock_extractor_run, mock_set_vara
):
    mock_abspath.return_value = '/path/to/yaml_file.yaml'
    mock_dirname.side_effect = lambda x: x
    mock_open.return_value.__enter__.return_value.read.return_value = 'abc'

    print("Before main")

    with patch.object(extractor, 'run') as mock_run:
        mock_run.return_value = Mock()

        print("Before main call")
        extractor.main()

        print("After main call")

    assert mock_set_vara.called
    assert mock_sap_config.called
    assert mock_metrics.called
    assert mock_extractor_run.called
    assert mock_open.called
    assert mock_run.called

 

 

 


Forum|alt.badge.img+6

Yeah I have tried by debugging after calling main function it’s not coming back.

from cognite.extractorutils import Extractor

def main() -> None:

    """

    Main entrypoint

    """

    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

    config_values_vault.set_vara()

    with Extractor(

        name="SAP_Extractor",

        description="An extractor to extract asset hierarchy from SAP based on root node",

        config_class=SapConfig,

        # version=__version__,

        # debugger

        version="1.0.0",

        run_handle=run,

        metrics=metrics,

        config_file_path= os.path.join(BASE_DIR, 'config.yaml'),

    ) as extractor:

        extractor.run()

 

The above is original code.


Anita Hæhre
Seasoned Practitioner
Forum|alt.badge.img+1
  • Head of Academy and Community
  • November 22, 2023

Hi Community! if there is someone here who can provide assistance on the above from @Harshita Sai we would greatly appreciate it. Your collective knowledge and experience make this community a valuable resource, so if you know how to help out here, we’d love to hear from you 🏅

 


Isha Thapliyal
Seasoned Practitioner
Forum|alt.badge.img+3

Hi @Harshita Sai 

Wanted to check in with you regarding the post you made a while back. We haven’t seen any updates, so I’m reaching out to see if everything has been resolved or if you still need assistance.

 

Looking forward to hearing from you!

Thanks


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings