Add TTP for even more difficult templates

[evilmonkey19]

Environment

• ntc-templates version: 5.0

Proposed Functionality

In some edge cases, TextFSM is not enough for capturing all the needed variables. For example, if a variable is nested inside another, then TextFSM cannot capture that. In such cases i suggest to use TTP as extra backup parser. This way it is possible to get more complex outputs.

Use Case

This is the case in the display ont info 0 1 2 3 in Huawei smartax in this lines:

 <T-CONT   4>          DBA Profile-ID:5
    <Gem Index 126>
    ------------------------------------------------------------------------
    |Serv-Type:ETH |Encrypt:off |Cascade:off |GEM-CAR:-            |
    |Upstream-priority-queue:-  |Downstream-priority-queue:-       |
    ------------------------------------------------------------------------
     Mapping VLAN  Priority Port   Port  Bundle  Flow  Transparent
     index                  type   ID    ID      CAR
    ------------------------------------------------------------------------
     0       500   -        -      -     -       -     -
    ------------------------------------------------------------------------
  <T-CONT   5>          DBA Profile-ID:5
    <Gem Index 126>
    ------------------------------------------------------------------------
    |Serv-Type:ETH |Encrypt:off |Cascade:off |GEM-CAR:-            |
    |Upstream-priority-queue:-  |Downstream-priority-queue:-       |
    ------------------------------------------------------------------------
     Mapping VLAN  Priority Port   Port  Bundle  Flow  Transparent
     index                  type   ID    ID      CAR
    ------------------------------------------------------------------------
     0       500   -        -      -     -       -     -
    ------------------------------------------------------------------------

[mjbear]

Hello @evilmonkey19 ,

Would you please further explain where there's a variable nested inside another?
Please provide example(s).

Thank you.

[evilmonkey19]

In this case the output would something as follows:

[{
"tcont_id": 4,
"dba_profile_id": 5,
"gem": {
  "gem_index": 126,
  "srv_type": "eth",
  "encrypt": "off",
  "cascade": "off",
  "gem-car": "",
  "upstream-priority-queue": "",
  "downstream-priority-queue": "",
  "vlans": [{
    "mapping_index": 0,
    "vlan": 500,
    "priority": "",
    "port_type": "",
    "port_id": "",
    "bundle_car": "",
    "flow": "",
    "transparent": "",
  }]
}
},
{
"tcont_id": 5,
"dba_profile_id": 5,
"gem": {
  "gem_index": 126,
  "srv_type": "eth",
  "encrypt": "off",
  "cascade": "off",
  "gem-car": "",
  "upstream-priority-queue": "",
  "downstream-priority-queue": "",
  "vlans": [{
    "mapping_index": 0,
    "vlan": 500,
    "priority": "",
    "port_type": "",
    "port_id": "",
    "bundle_car": "",
    "flow": "",
    "transparent": "",
  }]
}
}]

In this case you can see the nesting of the VLAN inside the GEM and the GEM inside the T-CONT. One T-CONT may have also multiple GEMs.

Currently, it is possible to have this level of complexity using a TTP parser. As I said before, I know that this case is pretty niche, but I have several devices similar nested outputs.

[jmcgill298]

@evilmonkey19 I understand your interest in having a more flexible output format, and that TextFSM is not able to provide that, however, we are not interested in taking on that additional work of maintaining TTP templates on top of what is being maintained here. For the issue that you have described above, I would say that you can restructure the output in post-processing.

[evilmonkey19]

I agree on the fact that it might be a burden for the project developing, testing and maintaining this feature. May I ask at least to put it as a possible solution the postprocessing for even more complex outputs using TTP? I was thinking somewhere like a FAQs, just to let beginners know what is possible and what not using TextFSM and NTC-Templates.

[mjbear]

In your example output, "gem" is a dictionary {}, not a list [] with a nested dictionary inside so I don't see how the tcont_id can have multiple gems.

I agree on the fact that it might be a burden for the project developing, testing and maintaining this feature. May I ask at least to put it as a possible solution the postprocessing for even more complex outputs using TTP? I was thinking somewhere like a FAQs, just to let beginners know what is possible and what not using TextFSM and NTC-Templates.

So what's being suggested is the NTC Templates project create a FAQ section to inform the community the existence of TTP, Genie, ciscoconfparse, ciscoconfparse2, etc (any I've missed)?

Though there already is a repo dedicated to documenting automation projects that @evilmonkey19 added updates to last month. 😀
(This thread is from June 17 and your merged PR on the above hyperlink was June 26.)

(I have no final say in this as a community voluneer, but) Would it be sufficient to possibly have ntc-templates docs link to the Awesome Network Automation repo which links out to other projects?
(TTP is already included in the awesome network automation README.md file.)

[evilmonkey19]

IMHO, what @mjbear suggested is even a better idea. NTC-templates is a great project for newcomers, therefore I think is worth it to point to the Awesome Network Automation repo 😄 .

[mjbear]

IMHO, what @mjbear suggested is even a better idea. NTC-templates is a great project for newcomers, therefore I think is worth it to point to the Awesome Network Automation repo 😄 .

Cool 😎

This may be something that could be added to docs/user/faq.md, but will anyone find it there?
💡 Better yet, maybe add a section to README.md for an "Additional Automation Resources" at the bottom (below the Questions section).

Example:

## Additional Automation Resources
There are situations where one solution or tool might not fulfill needs or as well as another. Fortunately there are often alternatives and the [Awesome Network Automation](https://github.com/networktocode/awesome-network-automation) list can help introduce you to additional resources and solutions!

Once we get a consensus I can pull together a PR.

Thoughts?
@evilmonkey19
@jmcgill298

[evilmonkey19]

I like it in the README.md file 💫

[mjbear]

@evilmonkey19
I submitted PR #1799