Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add navigation support for OmegaConfig Syntax #8

Open
Tracked by #58
noklam opened this issue May 9, 2024 · 1 comment
Open
Tracked by #58

Add navigation support for OmegaConfig Syntax #8

noklam opened this issue May 9, 2024 · 1 comment
Labels
feature Improvement of existing feature or new feature Python Pull requests that update Python code

Comments

@noklam
Copy link
Contributor

noklam commented May 9, 2024

Rough idea:

  • YAML return every config with a __line__ attribute
    • OmegaConf consume this __line__ metadata, then update the following constructors:
      • ContainerMetadata
      • OmegaConf.create
      • OmegaConf._create_impl
      • DictConfig.__init__
@noklam
Copy link
Contributor Author

noklam commented May 21, 2024

The pre-requisite for building this is to store the YAML line no/cursor position of the catalog entry at load-time.

For reference, ruamel.yaml provides the cursor information:

In [11]: from ruamel.yaml import YAML

In [12]: yaml = YAML()

In [13]: data = yaml.load("""
    ...:      # testing line and column based on SO
    ...:      # http://stackoverflow.com/questions/13319067/
    ...:      - key1: item 1
    ...:        key2: item 2
    ...:      - key3: another item 1
    ...:        key4: another item 2
    ...:         """)

In [14]: data
Out[14]: [{'key1': 'item 1', 'key2': 'item 2'}, {'key3': 'another item 1', 'key4': 'another item 2'}]

In [15]: type(data)
Out[15]: ruamel.yaml.comments.CommentedSeq

In [16]: data[0].lc
Out[16]: LineCol(3, 7)

In [17]: type(data[0])
Out[17]: ruamel.yaml.comments.CommentedMap

It was mentioned that some YAML loader preserve metadata about line and col from kedro-org/kedro#2821 (comment). After some investigation, it is less strictforward than expected. The example is a list, so you can do data[0].lc where data[0] is a CommentedSeq object. You can do similar things if the value is a dictionary. It doesn't work if it's a simple str or int. For example

# abc.yml
a: 1
b: 2
yaml = YAML()
with open("abc.yml") as f:
    data = yaml.load(f)

>>> data["a"]
1

In this case, data["a"] returns an int directly and all the metadata are lost.

@noklam noklam added the feature Improvement of existing feature or new feature label Sep 23, 2024
@noklam noklam moved this to Todo in Kedro VS Code Sep 25, 2024
@jitu5 jitu5 added the Python Pull requests that update Python code label Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Improvement of existing feature or new feature Python Pull requests that update Python code
Projects
No open projects
Status: Todo
Development

No branches or pull requests

2 participants