Mailing List Archive

Lyndon Nerenberg (VE7TFX/VE6BBM) wrote on 21/08/2023 22:14:
> Any chance somebody out there has a yacc grammar that will parse
> a Juniper config files? My immediate interest involves v19.X on
> our EX4300s, but anything in the ballpark would save me having to
> write one from scratch.

No need to reinvent that wheel:

root@foo> show configuration | display xml
root@foo> show configuration | display json

... then slurp into an ingestion engine in your favourite language.

Nick

Nick Hilliard writes:

> No need to reinvent that wheel:
>
> root@foo> show configuration | display xml
> root@foo> show configuration | display json

That doesn't quite work for this scenario. It would mean ssh-ing
to the switch to grab it, and that's pretty locked down. We already
have cron jobs running on the switches that tftp the config file
to a server, and I'd prefer to leverage off that.

Also, the yacc parser would let me do some validation checks
before we push new configs.

--lyndon

I would first try to understand what you are trying to achieve. JUNOS is
very flexible on this front and I am wondering why you think yacc is the
right way to achieve what you are trying to do.

If no one (or very few) these days is using yacc grammar for parsing router
configs, that should be a good indication whether you are on the right path
or not.

As already pointed out, navigating the XML trees or JSON structures
would be much easier than writing a yacc grammar parser.

Alternatively, I would look into the YANG files:
https://github.com/Juniper/yang

But without understanding what you are trying to do, this is just another
suggestion.

./diogo -montagner
JNCIE-SP 0x41A


On Tue, 22 Aug 2023 at 09:53, Lyndon Nerenberg (VE7TFX/VE6BBM) <
lyndon@orthanc.ca> wrote:

> Nick Hilliard writes:
>
> > No need to reinvent that wheel:
> >
> > root@foo> show configuration | display xml
> > root@foo> show configuration | display json
>
> That doesn't quite work for this scenario. It would mean ssh-ing
> to the switch to grab it, and that's pretty locked down. We already
> have cron jobs running on the switches that tftp the config file
> to a server, and I'd prefer to leverage off that.
>
> Also, the yacc parser would let me do some validation checks
> before we push new configs.
>
> --lyndon
>

>
> We already
> have cron jobs running on the switches that tftp the config file
> to a server, and I'd prefer to leverage off that.
>

Use the same cron jobs to save a copy as xml/json and pull the file off.

On Mon, Aug 21, 2023 at 7:52?PM Lyndon Nerenberg (VE7TFX/VE6BBM) <
lyndon@orthanc.ca> wrote:

> Nick Hilliard writes:
>
> > No need to reinvent that wheel:
> >
> > root@foo> show configuration | display xml
> > root@foo> show configuration | display json
>
> That doesn't quite work for this scenario. It would mean ssh-ing
> to the switch to grab it, and that's pretty locked down. We already
> have cron jobs running on the switches that tftp the config file
> to a server, and I'd prefer to leverage off that.
>
> Also, the yacc parser would let me do some validation checks
> before we push new configs.
>
> --lyndon
>

Sorry, I don't know the answer to your initial inquiry

| We already have cron jobs running on the switches that tftp the config file to a server, and I'd prefer to leverage off that.

Perhaps an easy interim adaptation would be to modify your scheduled job
to tftp the latest xml or json then clean it up?

root@BB6> ...on | display xml | save /config/junos-config-latest.xml
Wrote 139 lines of output to '/config/junos-config-latest.xml'

root@BB6> ...n | save /config/junos-config-latest.json
Wrote 169 lines of output to '/config/junos-config-latest.json'

root@BB6>
root@BB6> file list /config/ | grep latest
junos-config-latest.json
junos-config-latest.xml

root@BB6>

Hope you find the final solution, as the parser checks sound worth the
squeeze.

Diogo Montagner writes:
> --000000000000728632060377d0b2
> Content-Type: text/plain; charset="UTF-8"
>
> I would first try to understand what you are trying to achieve. JUNOS is
> very flexible on this front and I am wondering why you think yacc is the
> right way to achieve what you are trying to do.

Because I've been writing yacc grammars for decades. I just wanted to
see if someone had already done it, as that would save me some time.
But if there's nothing out there I'll just roll one myself.

--lyndon

On Tue, 22 Aug 2023 at 03:30, Lyndon Nerenberg (VE7TFX/VE6BBM)
<lyndon@orthanc.ca> wrote:

> Because I've been writing yacc grammars for decades. I just wanted to
> see if someone had already done it, as that would save me some time.
> But if there's nothing out there I'll just roll one myself.

I sympathise with your problem and I've always wanted vendors to
publish their parsers, there are many use cases.

But as such does not exist, this avenue of attack seems very
problematic, unless this whole network lives and dies with you. If
not, then your feature velocity now depends on someone adding support
for new keywords to the parser generator, no one who comes after you
will thank you for adding this dependency to the process. But they
might call you and pay stupid money for a 5 min job, so maybe it is a
great idea.

--
++ytti

Lyndon Nerenberg (VE7TFX/VE6BBM) wrote on 22/08/2023 01:27:
> Because I've been writing yacc grammars for decades. I just wanted to
> see if someone had already done it, as that would save me some time.
> But if there's nothing out there I'll just roll one myself.

check out xorp and vyos - both contain code to parse junos style
configurations. Just bear in mind that they provide basic tokeniser
functionality, which parses the configurations into token trees. The
config interpretation can then be handled on a modular basis.

Nick

I agree with the yang approach. I would probably convert to using Yang and
download the yang files for whatever code version you are running. For
every code version you can download the yang files.
Then you have a choice to decide, either right a yang to yacc conversion
tool or find one, or think about switching to a netconf approach or
ODL/restconf -> netconf style approach depending on how you are using the
yacc files today as that will auto check and verify basics and reject non
compliant configs to some level.


On Tue, Aug 22, 2023 at 9:36?AM Diogo Montagner <diogo.montagner@gmail.com>
wrote:

> I would first try to understand what you are trying to achieve. JUNOS is
> very flexible on this front and I am wondering why you think yacc is the
> right way to achieve what you are trying to do.
>
> If no one (or very few) these days is using yacc grammar for parsing
> router configs, that should be a good indication whether you are on the
> right path or not.
>
> As already pointed out, navigating the XML trees or JSON structures
> would be much easier than writing a yacc grammar parser.
>
> Alternatively, I would look into the YANG files:
> https://github.com/Juniper/yang
>
> But without understanding what you are trying to do, this is just another
> suggestion.
>
> ./diogo -montagner
> JNCIE-SP 0x41A
>
>
> On Tue, 22 Aug 2023 at 09:53, Lyndon Nerenberg (VE7TFX/VE6BBM) <
> lyndon@orthanc.ca> wrote:
>
>> Nick Hilliard writes:
>>
>> > No need to reinvent that wheel:
>> >
>> > root@foo> show configuration | display xml
>> > root@foo> show configuration | display json
>>
>> That doesn't quite work for this scenario. It would mean ssh-ing
>> to the switch to grab it, and that's pretty locked down. We already
>> have cron jobs running on the switches that tftp the config file
>> to a server, and I'd prefer to leverage off that.
>>
>> Also, the yacc parser would let me do some validation checks
>> before we push new configs.
>>
>> --lyndon
>>
>

OFFS people, spare me the bikeshed. It was a simple yes/no question.
In case you missed it, here is the decision tree:


/-------\
| START |
\-------/
|
|
^
/ \
/ \
/ \
/ \
/ \
/ Do You \ No /------\
< Have A > --------- | STOP |
\ Yacc / \------/
\ Grammar? /
\ /
\ /
\ /
\ /
\ /
| Yes
|
^
/ \
/ \
/ \
/ \
/ \ No /------\
< Share? > --------- | STOP |
\ / \------/
\ /
\ /
\ /
\ /
| Yes
|
|----------|
| Email |
| Grammar |
|----------|
|
|
/------\
| STOP |
\------/


--lyndon

Good luck with your pursuit with that attitude.

On Tue, Aug 22, 2023 at 9:48?AM Lyndon Nerenberg (VE7TFX/VE6BBM) <
lyndon@orthanc.ca> wrote:

> OFFS people, spare me the bikeshed. It was a simple yes/no question.
> In case you missed it, here is the decision tree:
>
>
> /-------\
> | START |
> \-------/
> |
> |
> ^
> / \
> / \
> / \
> / \
> / \
> / Do You \ No /------\
> < Have A > --------- | STOP |
> \ Yacc / \------/
> \ Grammar? /
> \ /
> \ /
> \ /
> \ /
> \ /
> | Yes
> |
> ^
> / \
> / \
> / \
> / \
> / \ No /------\
> < Share? > --------- | STOP |
> \ / \------/
> \ /
> \ /
> \ /
> \ /
> | Yes
> |
> |----------|
> | Email |
> | Grammar |
> |----------|
> |
> |
> /------\
> | STOP |
> \------/
>
>
> --lyndon
>


--
Brent Jones
brent@brentrjones.com

On 8/21/23 7:09?PM, Diogo Montagner wrote:
> I would first try to understand what you are trying to achieve. JUNOS is
> very flexible on this front and I am wondering why you think yacc is the
> right way to achieve what you are trying to do.

Drive by comment:

Perhaps the OP is trying to parse a (pile of) config file(s) downstream
of the generation thereof and has no ability to alter their generation.



Grant. . . .

On Tue, Aug 22, 2023 at 11:39?PM Grant Taylor via NANOG <nanog@nanog.org> wrote:
>
> On 8/21/23 7:09?PM, Diogo Montagner wrote:
> > I would first try to understand what you are trying to achieve. JUNOS is
> > very flexible on this front and I am wondering why you think yacc is the
> > right way to achieve what you are trying to do.
>
> Drive by comment:
>
> Perhaps the OP is trying to parse a (pile of) config file(s) downstream
> of the generation thereof and has no ability to alter their generation.

this is a common problem (or is common when I look at things, perhaps
I'm looking wrongly, but...)
I'd love to have something that parsed all of my device type configs
and output the results into a
'database' that i could then ask questions of like:
"Hey, what NTP servers are configured on all devices?"
"Hey, which devices have this <access-list/firewall/user> configured on them?"

There are a host of other things I could ask those are but 2 simple
examples, and YES I can
grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I
provided... but really
that's NOT the way I want to do this, and I do really have a bunch of
other questions I'd
like to ask, regularly, to solve rollout-of-new-feature / compliance /
legal / troubleshooting / etc
questions.

In looking around there are examples of some of this, in a way, the
most common thing
I end up looking at, and getting sad about, is some java monstrosity
(who's name escapes me)
but has shown up in a few nanog presentations over the years... it
makes me sad because it's
not super useful in my world :( 'hard to use' is probably the best
way to describe it.

One note about XML and Juniper, the schema changes by OS version, it
changes quite a bit :(
You CAN parse through it reasonably well with python lxml.Etree,
because (I think) python's parse
is VERY forgiving. If you attempt this path with golang :( you will be
sad, very sad :( because
the go->xml world is very 'build a struct of structs that mirrors the
xml tree' and 'changes at every
OS version' means now you have a LOT of versions of that :(
maintenance gets back to saku's
comment about feature velocity :(

I do see:
https://pypi.org/project/juniper-nxpy/

which may be useful to you as well Lyndon.
(I'd also point to tftp as not being the super best option from a
security and reliability perspective,
but if that's what you've got that's what you've got... you COULD have
the switch cronjobs curl/post
to an https destination with little hard work, and a gain in
reliabilty/security)

-chris

Christopher Morrow wrote:
>
> In looking around there are examples of some of this, in a way, the
> most common thing
> I end up looking at, and getting sad about, is some java monstrosity
> (who's name escapes me)
> but has shown up in a few nanog presentations over the years... it
> makes me sad because it's
> not super useful in my world :( 'hard to use' is probably the best
> way to describe it.
>
>
You're probably thinking of Batfish.

Justin H.

FWIW, I find the config archival on-commit to be a very useful feature. I use it with SCP. Sadly, neither J nor C support doing this with an SSH private key on the router, you have to use a password. J at least encrypts the password if you do it right (…”URL” password “plain-text”). Cisco does not encrypt this password in the config (perhaps they do in more recent releases. All my C hardware is ancient.

C will do it on “write”. Not quite as good as “on-commit”, but since C lacks commit, is what it is.

YMMV.

Owen


> On Aug 24, 2023, at 07:11, Christopher Morrow <morrowc.lists@gmail.com> wrote:
>
> ?On Tue, Aug 22, 2023 at 11:39?PM Grant Taylor via NANOG <nanog@nanog.org> wrote:
>>
>>> On 8/21/23 7:09?PM, Diogo Montagner wrote:
>>> I would first try to understand what you are trying to achieve. JUNOS is
>>> very flexible on this front and I am wondering why you think yacc is the
>>> right way to achieve what you are trying to do.
>>
>> Drive by comment:
>>
>> Perhaps the OP is trying to parse a (pile of) config file(s) downstream
>> of the generation thereof and has no ability to alter their generation.
>
> this is a common problem (or is common when I look at things, perhaps
> I'm looking wrongly, but...)
> I'd love to have something that parsed all of my device type configs
> and output the results into a
> 'database' that i could then ask questions of like:
> "Hey, what NTP servers are configured on all devices?"
> "Hey, which devices have this <access-list/firewall/user> configured on them?"
>
> There are a host of other things I could ask those are but 2 simple
> examples, and YES I can
> grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I
> provided... but really
> that's NOT the way I want to do this, and I do really have a bunch of
> other questions I'd
> like to ask, regularly, to solve rollout-of-new-feature / compliance /
> legal / troubleshooting / etc
> questions.
>
> In looking around there are examples of some of this, in a way, the
> most common thing
> I end up looking at, and getting sad about, is some java monstrosity
> (who's name escapes me)
> but has shown up in a few nanog presentations over the years... it
> makes me sad because it's
> not super useful in my world :( 'hard to use' is probably the best
> way to describe it.
>
> One note about XML and Juniper, the schema changes by OS version, it
> changes quite a bit :(
> You CAN parse through it reasonably well with python lxml.Etree,
> because (I think) python's parse
> is VERY forgiving. If you attempt this path with golang :( you will be
> sad, very sad :( because
> the go->xml world is very 'build a struct of structs that mirrors the
> xml tree' and 'changes at every
> OS version' means now you have a LOT of versions of that :(
> maintenance gets back to saku's
> comment about feature velocity :(
>
> I do see:
> https://pypi.org/project/juniper-nxpy/
>
> which may be useful to you as well Lyndon.
> (I'd also point to tftp as not being the super best option from a
> security and reliability perspective,
> but if that's what you've got that's what you've got... you COULD have
> the switch cronjobs curl/post
> to an https destination with little hard work, and a gain in
> reliabilty/security)
>
> -chris

On Thu, Aug 24, 2023 at 10:10 AM, Christopher Morrow <
morrowc.lists@gmail.com> wrote:

> On Tue, Aug 22, 2023 at 11:39?PM Grant Taylor via NANOG <nanog@nanog.org>
> wrote:
>
> On 8/21/23 7:09?PM, Diogo Montagner wrote:
>
> I would first try to understand what you are trying to achieve. JUNOS is
> very flexible on this front and I am wondering why you think yacc is the
> right way to achieve what you are trying to do.
>
> Drive by comment:
>
> Perhaps the OP is trying to parse a (pile of) config file(s) downstream of
> the generation thereof and has no ability to alter their generation.
>
> this is a common problem (or is common when I look at things, perhaps I'm
> looking wrongly, but...)
> I'd love to have something that parsed all of my device type configs and
> output the results into a
> 'database' that i could then ask questions of like:
> "Hey, what NTP servers are configured on all devices?"
> "Hey, which devices have this <access-list/firewall/user> configured on
> them?"
>
>
>

Isn't this YANG/NETCONF, and squish it all into DB/directory full of files?

Basically a more standardized format for representing device configurations
/ states?

W


There are a host of other things I could ask those are but 2 simple
> examples, and YES I can
> grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I
> provided... but really
> that's NOT the way I want to do this, and I do really have a bunch of
> other questions I'd
> like to ask, regularly, to solve rollout-of-new-feature / compliance /
> legal / troubleshooting / etc
> questions.
>
> In looking around there are examples of some of this, in a way, the most
> common thing
> I end up looking at, and getting sad about, is some java monstrosity
> (who's name escapes me)
> but has shown up in a few nanog presentations over the years... it makes
> me sad because it's
> not super useful in my world :( 'hard to use' is probably the best way to
> describe it.
>
> One note about XML and Juniper, the schema changes by OS version, it
> changes quite a bit :(
> You CAN parse through it reasonably well with python lxml.Etree, because
> (I think) python's parse
> is VERY forgiving. If you attempt this path with golang :( you will be
> sad, very sad :( because
> the go->xml world is very 'build a struct of structs that mirrors the xml
> tree' and 'changes at every
> OS version' means now you have a LOT of versions of that :( maintenance
> gets back to saku's
> comment about feature velocity :(
>
> I do see:
> https://pypi.org/project/juniper-nxpy/
>
> which may be useful to you as well Lyndon.
> (I'd also point to tftp as not being the super best option from a security
> and reliability perspective,
> but if that's what you've got that's what you've got... you COULD have the
> switch cronjobs curl/post
> to an https destination with little hard work, and a gain in
> reliabilty/security)
>
> -chris
>

It may have been covered already, but another place to look is at code for
Batfish

https://github.com/batfish/batfish

https://www.batfish.org/

Its goal. and there are even podcast episodes that cover it:

https://www.podchaser.com/podcasts/episode-archive-packet-pushers-1755/episodes/heavy-networking-658-using-bat-156733916

https://blog.ipspace.net/2019/09/intent-based-networking-with-batfish-on.html

was to parse configs and out of that derive overall topology forwarding
graph and do config verification, so the tool itself may be of some use,
but for their parsing they use ANTLR, so you can reuse some of the code.

Which would be a pretty useful functionality, because then you could build
router stack independent policy processors and verification.

And it would be nice to have RC9067 policy parser and compiler into
proprietary config schema of one's choice. With some verification.

Yan



On Thu, Aug 24, 2023 at 1:04?PM Warren Kumari <warren@kumari.net> wrote:

>
>
>
>
> On Thu, Aug 24, 2023 at 10:10 AM, Christopher Morrow <
> morrowc.lists@gmail.com> wrote:
>
>> On Tue, Aug 22, 2023 at 11:39?PM Grant Taylor via NANOG <nanog@nanog.org>
>> wrote:
>>
>> On 8/21/23 7:09?PM, Diogo Montagner wrote:
>>
>> I would first try to understand what you are trying to achieve. JUNOS is
>> very flexible on this front and I am wondering why you think yacc is the
>> right way to achieve what you are trying to do.
>>
>> Drive by comment:
>>
>> Perhaps the OP is trying to parse a (pile of) config file(s) downstream
>> of the generation thereof and has no ability to alter their generation.
>>
>> this is a common problem (or is common when I look at things, perhaps I'm
>> looking wrongly, but...)
>> I'd love to have something that parsed all of my device type configs and
>> output the results into a
>> 'database' that i could then ask questions of like:
>> "Hey, what NTP servers are configured on all devices?"
>> "Hey, which devices have this <access-list/firewall/user> configured on
>> them?"
>>
>>
>>
>
> Isn't this YANG/NETCONF, and squish it all into DB/directory full of files?
>
> Basically a more standardized format for representing device
> configurations / states?
>
> W
>
>
> There are a host of other things I could ask those are but 2 simple
>> examples, and YES I can
>> grep/sed/awk|sort|uniq|sort-rn my way to success for the 2 examples I
>> provided... but really
>> that's NOT the way I want to do this, and I do really have a bunch of
>> other questions I'd
>> like to ask, regularly, to solve rollout-of-new-feature / compliance /
>> legal / troubleshooting / etc
>> questions.
>>
>> In looking around there are examples of some of this, in a way, the most
>> common thing
>> I end up looking at, and getting sad about, is some java monstrosity
>> (who's name escapes me)
>> but has shown up in a few nanog presentations over the years... it makes
>> me sad because it's
>> not super useful in my world :( 'hard to use' is probably the best way to
>> describe it.
>>
>> One note about XML and Juniper, the schema changes by OS version, it
>> changes quite a bit :(
>> You CAN parse through it reasonably well with python lxml.Etree, because
>> (I think) python's parse
>> is VERY forgiving. If you attempt this path with golang :( you will be
>> sad, very sad :( because
>> the go->xml world is very 'build a struct of structs that mirrors the xml
>> tree' and 'changes at every
>> OS version' means now you have a LOT of versions of that :( maintenance
>> gets back to saku's
>> comment about feature velocity :(
>>
>> I do see:
>> https://pypi.org/project/juniper-nxpy/
>>
>> which may be useful to you as well Lyndon.
>> (I'd also point to tftp as not being the super best option from a
>> security and reliability perspective,
>> but if that's what you've got that's what you've got... you COULD have
>> the switch cronjobs curl/post
>> to an https destination with little hard work, and a gain in
>> reliabilty/security)
>>
>> -chris
>>
>
>

On Thu, Aug 24, 2023 at 11:10?AM Justin H. <justindh.ml@gmail.com> wrote:
>
> Christopher Morrow wrote:
> >
> > In looking around there are examples of some of this, in a way, the
> > most common thing
> > I end up looking at, and getting sad about, is some java monstrosity
> > (who's name escapes me)

> You're probably thinking of Batfish.

yes! I kept wanting to say Bee... something :) thanks!

On Thu, Aug 24, 2023 at 11:45?AM Warren Kumari <warren@kumari.net> wrote:
>> from: me
>> this is a common problem (or is common when I look at things, perhaps I'm looking wrongly, but...)
>> I'd love to have something that parsed all of my device type configs and output the results into a
>> 'database' that i could then ask questions of like:
>> "Hey, what NTP servers are configured on all devices?"
>> "Hey, which devices have this <access-list/firewall/user> configured on them?"
>>
>>
>
> Isn't this YANG/NETCONF, and squish it all into DB/directory full of files?
>

yup, sort of... I think the yang parts are still not 100% there, and
also then I'd need to do the
1 thing I've been avoiding (aside from printing from a unix machine)
so far: "learn yang things" :)

> Basically a more standardized format for representing device configurations / states?

yup! really any way that would be relatively steady over time to go from:
"router config on router over there ->"
to:
"now in some useful way stored in a 'database'"

was/is my goal... or one goal of the many goals I attempt to get to a
solution for every once in a while :)