Can't import api from exos

  • 0
  • 1
  • Problem
  • Updated 1 month ago
  • Solved
  • (Edited)
Hey everybody!

I'm working on a project with an Extreme Networks Summit X440-8p running firmware-version 16.1.1.4. I want to write a python script, but I'm stuck at the beginning.

The Python Scripting Guide (http://www.extremenetworks.com/wp-content/uploads/2015/02/Python_Getting_Started_Guide.pdf) tells me I have to import api from exos with the following line:

"from exos import api"

Unfortunately, every time I want to execute my script (which just has a print command at the moment), there's an error:

"* X440-8p.81 # run script print_test
Traceback (most recent call last):
  File "/config/print_test.py", line 1, in <module>
    from exos import api
  File "/exos/tools/lib/python2.7/site-packages/exos/api/__init__.py", line 21, in <module>
  File "/exos/tools/lib/python2.7/site-packages/exos/api/ems.py", line 10, in <module>
ImportError: No module named _exos_ext_ems".

What am I doing wrong? It's exactly how the scripting guide tells me to import the api. This site tells me the same: http://documentation.extremenetworks.com/python/

Has anything changed since it looks like this site is for software-version 15.7.1 whereas I'm running 16.1.1.4?

Thanks in advance!
Mirco
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb

Posted 3 years ago

  • 0
  • 1
Photo of Stephen Williams

Stephen Williams, Employee

  • 9,076 Points 5k badge 2x thumb
Marco,

I will look into the problem with loading the API, but you don't need to use the API to do simple python scripts.  You can print "hello" by using the text below and run the python script.

print ("hello")

output:
Switch# run script test
hello
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb
Hello Stephen,

thanks for your reply and for looking into this issue!

I know that I can run simple scripts without loading the API but our aim isn't just printing something.

Actually, we want to read various parameters such as stream information (bandwidth etc.), and write them to a file, which we can read on a PC to display those data. We thought about using CLI-commands to achieve this. Therefore, we need the API.

Thanks again, hope you'll get back soon :)

Mirco
(Edited)
Photo of Dave Hammers

Dave Hammers, Dir SW Engineering

  • 3,802 Points 3k badge 2x thumb

Hi Mirco,

EXOS has 2 python environments. one for writing scripts and the other for writing python apps.

Python that run with 'run script <name>' use the exsh library.

The python call you want is:

cliResult = exsh.clicmd(cmd, capture=True, xml=False)

If you wanted to capture the data in XML format instead of formatted CLI output:

xmlResult = exsh.clicmd(cmd, capture=False, xml=True)

Sometimes XML is easier to use to extract data instead of parsing the CLI output. CLI output format  could change from release to release.

Here is a quick example with a pretty print script named xmlpp.py:

import xml.dom.minidom

xmlResult = exsh.clicmd('show ports statistics no-refresh', xml=True).replace('</reply>','</reply>\n')
for line in xmlResult.splitlines():
    a = xml.dom.minidom.parseString(line)
    print a.toprettyxml(indent=' ' * 4)


run script xmlpp.py


<?xml version="1.0" ?>
<reply>
    <status>MORE</status>
    <message>
        <show_ports_stats>
            <port>1</port>
            <portNoSnmp>1</portNoSnmp>
            <dot1dTpPortInDiscards>0</dot1dTpPortInDiscards>
            <dot1dTpPortInFrames>1869</dot1dTpPortInFrames>
            <dot1dTpPortMaxInfo>1500</dot1dTpPortMaxInfo>
            <dot1dTpPortOutFrames>1868</dot1dTpPortOutFrames>
            <linkState>1</linkState>
            <portList>1-34</portList>
            <rxBcast>0</rxBcast>
            <rxByteCnt>608062</rxByteCnt>
            <rxMcast>937</rxMcast>
            <rxPktCnt>1869</rxPktCnt>
            <txBcast>0</txBcast>
            <txByteCnt>334260</txByteCnt>
            <txMcast>935</txMcast>
            <txPktCnt>1868</txPktCnt>
        </show_ports_stats>
    </message>
</reply>

<?xml version="1.0" ?>
<reply>
    <status>MORE</status>
    <message>
        <show_ports_stats>
            <port>2</port>
            <portNoSnmp>2</portNoSnmp>
            <dot1dTpPortInDiscards>0</dot1dTpPortInDiscards>
            <dot1dTpPortInFrames>0</dot1dTpPortInFrames>
            <dot1dTpPortMaxInfo>1500</dot1dTpPortMaxInfo>
            <dot1dTpPortOutFrames>0</dot1dTpPortOutFrames>
            <linkState>0</linkState>
            <portList>1-34</portList>
            <rxBcast>0</rxBcast>
            <rxByteCnt>0</rxByteCnt>
            <rxMcast>0</rxMcast>
            <rxPktCnt>0</rxPktCnt>
            <txBcast>0</txBcast>
            <txByteCnt>0</txByteCnt>
            <txMcast>0</txMcast>
            <txPktCnt>0</txPktCnt>
        </show_ports_stats>
    </message>
</reply>

<?xml version="1.0" ?>
<reply>
    <status>MORE</status>
    <message>
        <show_ports_stats>
            <port>3</port>
            <portNoSnmp>3</portNoSnmp>
            <dot1dTpPortInDiscards>0</dot1dTpPortInDiscards>
            <dot1dTpPortInFrames>0</dot1dTpPortInFrames>
            <dot1dTpPortMaxInfo>1500</dot1dTpPortMaxInfo>
            <dot1dTpPortOutFrames>0</dot1dTpPortOutFrames>
            <linkState>0</linkState>
            <portList>1-34</portList>
            <rxBcast>0</rxBcast>
            <rxByteCnt>0</rxByteCnt>
            <rxMcast>0</rxMcast>
            <rxPktCnt>0</rxPktCnt>
            <txBcast>0</txBcast>
            <txByteCnt>0</txByteCnt>
            <txMcast>0</txMcast>
            <txPktCnt>0</txPktCnt>
        </show_ports_stats>
    </message>
</reply>
etc ...


DaveH

 

Photo of Stephen Williams

Stephen Williams, Employee

  • 9,076 Points 5k badge 2x thumb
I want to add to Dave's post.  I created a script that will pull the ports, RX and TX bandwidth and display the data.

The function called cmd2data is used to make CLI output into a table of dictionaries.

Calling cmd2data function to run 'show port utilization bandwidth':
print cmd2data('show port utilization bandwidth')
Output of the print command above is displayed below for one port only as an example:
{'show_ports_utilization': {'linkSpeed': 0, 'rxPeakBytesPerSec': 0, 'portNoSnmp': 0, 'txBytesPerSec': 0, 'portList': '1-54', 'rxPktsPerSec': 0, 'txPeakBytesPerSec': 0, 'txBwPeakPercent': '0.000000', 'rxBytesPerSec': 0, 'linkState': 0, 'rxBwPercent': '0.000000', 'samplingTime': 15, 'rxPeakPktsPerSec': 0, 'rxBwPeakPercent': '0.000000', 'txPeakPktsPerSec': 0, 'txBwPercent': '0.000000', 'txPktsPerSec': 0, 'port': 1}

The show_ports function grabs the data you want from the output that you want and prints it.  You could add to the script to tftp the data to a tftp server.  Hope this helps. 

Python Script.
def cmd2data(clicmd):
    import re
    import xml.etree.cElementTree as ElementTree
    re_reply = re.compile(r'<reply>.+?</reply>', re.DOTALL)
    xmlout = exsh.clicmd(clicmd, capture=False, xml=True)
    data = []
    for reply in re.finditer(re_reply, xmlout):
        if reply:
            reply_xml = reply.group()
            root = ElementTree.fromstring(reply_xml)
            for message in root.iter('message'):
                for element in message:
                    mdata = {}
                    edata = {}
                    for e in element:
                        text = int(e.text) if e.text is not None and e.text.isdigit() else e.text
                        edata[e.tag] = text
                    mdata[element.tag] = edata
                    data.append(mdata)
    return data



def show_ports():
    port_util = cmd2data('show port utilization bandwidth')
    port_table = {}

    for line in port_util:
        port = line['show_ports_utilization']['port']
        rx_bw = line['show_ports_utilization']['rxBwPercent']
        tx_bw = line['show_ports_utilization']['txBwPercent']
        print ("Port Number: %s --- RX BW:  %s --- TX BW:  %s") % (port,rx_bw,tx_bw)
        print ("__________________________________________________________")
    return port_table

show_ports()

Script output:

# run script portutil
Port Number: 1 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 2 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 3 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 4 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 5 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 6 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 7 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 8 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 9 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
Port Number: 10 --- RX BW:  0.000000 --- TX BW:  0.000000
_______________________________________________________________
(Edited)
Photo of Ryan Mathews

Ryan Mathews, Alum

  • 8,988 Points 5k badge 2x thumb
This is impressive Stephen.  Thanks.

Python in EXOS....very powerful together!
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb
Hi Dave,thanks for your comment - the code works like a charm :) Unfornutaley, we would like to run this script periodically without any user-interaction. Therefore, we would like to write a native python app which we yould start as a process.

We tried to do a print output to the console with the code-example in the python scripting guide (http://www.extremenetworks.com/wp-content/uploads/2015/02/Python_Getting_Started_Guide.pdf)

After we created the process with "create process test python-module test start on-demand" and started it with "start process test" it just says "Started test successfully". No print-output to the console at all... We even tried to run the example in the python scripting guide, but it just said "Buffer "MyBuffer" not found in MyFirstApp"

Can you give us any advice what we're doing wrong?

Thanks in advance!
Mirco
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb
Ok, so i just figured out that the output won't be printed to the telnet-session-CLI but to the native CLI. Because that worked, I tried to execute a CLI command (show vlan), but it printed "command parse error" and "This command ("h") is not supported using execCLI XML API"

This is our code:
import sys
import logging
from exos import api

# Setup Logging
logger = logging.getLogger('xml_reader_process')
logger.setLevel(logging.DEBUG)
logHandler = api.TraceBufferHandler("MyBuffer", 5120)
logHandler.setLevel(logging.DEBUG)
logHandler.setFormatter(logging.Formatter("%(levelname)s:%(threadName)s:%(name)s:%(funcName)s.%(lineno)s:: %(message)s"))
logger.addHandler(logHandler)

try:

    def main():
        if not hasattr(sys, 'expy') or not sys.expy:
            print "Must be run within expy"
            return

        print "Main started"
        output = api.exec_cli("show vlan",5)
        print output

main()

except BaseException, e:
    logger.error("Exception on startup, {}".format(e), exc_info=True)

And this is the output:
Main started
Command parse error
This command ("h") is not supported using execCLI XML API.
Command parse error
Command parse error
Command parse error
Command parse error
Command parse error
Command parse error
Command parse error
I couldn't find a list of the supported commands, but I thought every CLI command would be supported - except those that has to be prompted?

It would be great if someone could have a look at our code and could explain what we're doing wrong!

Thanks so much :)
Mirco
(Edited)
Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 13,676 Points 10k badge 2x thumb
Hi,

api.exec_cli() needs a dictionary as a first argument, not a string.

cmd = ['show vlan']
output = api.exec_cli(cmd,5)

should work.
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb
Hi Stephane,

thanks a lot - it worked! I thought we would need a string because it's stated in the python scripting guide:
exos.api.exec_cli(cmds, timeout=0)

Send a list of commands to the CLI and return the output. This call will block until all commands have completed. This is a non-interactive, session-less version of the CLI. It doesn’t prompt or page, meaning some CLI commands are not valid in it (example: dis clipaging).

Parameters:
cmds (str): a list of commands.
timeout (int): timeout value, defaults to 0.

Returns:
A string of the CLI output (str).
So, we're now getting the formatted CLI output - is there a way to get it as XML like in the other python environment? As Dave stated, the CLI output could change from release to release, so it would be great to get XML instead of formatted CLI output.
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb
Hi Dave, thanks for your suggestion!

I thought about it, but unfortunately, this won't help us :( 

Here's what we're trying to achieve:
We want to write two python apps. The first one should run on the switch and collect specific data like avb status, stream information and so on.

The second app should run on a PC and connect to the app that's running on the switch to get the collected data and display it for monitoring purposes.
This should be possible in the app-environment with socket-programming, right? 

We know, there are already monitoring tools which we could use, but we want to write our own app to adapt it exactly to our needs.

But you say there's no way to get XML-formatted data in the app-environment? That would be too bad :(

If that's the case, the only way I can think of is the following: connect our monitoring-app to the switch via telnet, run a script, let's say every 30 seconds, and capture the XML-output.
That wouldn't be as nice as a native app on the switch, but it should work, right?

If you have any other ideas, please let me know! (a way to get XML-formatted data in the app-environment would be great ;))

Thanks a lot for your great support!
Photo of Dave Hammers

Dave Hammers, Dir SW Engineering

  • 3,792 Points 3k badge 2x thumb

I have one other idea. There is a script included with EXOS: cli2json.py.

This script runs any EXOS command, captures the XML and converts it to JSON.

JSON may be easier to process since python has simple conversion routines that converts JSON to/from dictionaries/lists.

From the app environment, spawn an EXOS shell to run the cli2json.py script with the desired CLI command and collect the data in JSON. An example python snippet is below.

You can run cli2json.py from the cli prompt to see what kind of information a show command will give you before including it in a script. My recommendation is to run it from a telnet session instead of the console since it can display a lot of data and console speeds are slow.

Be sure to use the -d when running it from a CLI prompt so the output is human readable.

It is important not to use the 'auto refresh' capabilities of some of the EXOS commands. E.g. show ports will display a list of ports and refresh it on the screen. If it is called from cli2json.py, the script will never return.

The cli2json.py JSON output also has a variable CLIoutput which contains what the actual CLI display would have been for the command. This way you get the formatted output as well as the data structures that were used to create the display 

The non-refreshing format of that command would look like: show ports no-refresh

From a CLI prompt:
run script cli2json.py -d <show command>

E.g.

run script cli2json.py -d show port 1-2 statistics no-refresh
[
  {
    "CLIoutput": "Port      Link       Tx Pkt     Tx Byte      Rx Pkt     Rx Byte      Rx Pkt      Rx Pkt      Tx Pkt      Tx Pkt\n          State       Count       Count       Count       Count       Bcast       Mcast       Bcast       Mcast\n========= ===== =========== =========== =========== =========== =========== =========== =========== ===========\n1         A             128       22980         128       22599           0          65           0          63\n2         R               0           0           0           0           0           0           0           0\n========= ===== =========== =========== =========== =========== =========== =========== =========== ===========\n          > in Port indicates Port Display Name truncated past 8 characters\n          > in Count indicates value exceeds column width. Use 'wide' option or '0' to clear.\n          Link State: A-Active, R-Ready, NP-Port Not Present, L-Loopback\n"
  },
  {
    "show_ports_stats": {
      "dot1dTpPortInDiscards": 0,
      "dot1dTpPortInFrames": 128,
      "dot1dTpPortMaxInfo": 1500,
      "dot1dTpPortOutFrames": 128,
      "linkState": 1,
      "port": 1,
      "portList": "1-2",
      "portNoSnmp": 1,
      "rxBcast": 0,
      "rxByteCnt": 22599,
      "rxMcast": 65,
      "rxPktCnt": 128,
      "txBcast": 0,
      "txByteCnt": 22980,
      "txMcast": 63,
      "txPktCnt": 128
    },
    "status": "MORE"
  },
  {
    "show_ports_stats": {
      "dot1dTpPortInDiscards": 0,
      "dot1dTpPortInFrames": 0,
      "dot1dTpPortMaxInfo": 1500,
      "dot1dTpPortOutFrames": 0,
      "linkState": 0,
      "port": 2,
      "portList": "1-2",
      "portNoSnmp": 2,
      "rxBcast": 0,
      "rxByteCnt": 0,
      "rxMcast": 0,
      "rxPktCnt": 0,
      "txBcast": 0,
      "txByteCnt": 0,
      "txMcast": 0,
      "txPktCnt": 0
    },
    "status": "SUCCESS"
  }
]


Now for the scripting part.

Spawning an EXOS shell from inside the app environment enables the app to run the cli2json.py script and collect the JSON formatted result.

import subprocess
import json


def exos2json(cmd):
    JSONERROR = 'error'
    shellCmd = '/exos/bin/exsh -n 0 -b -c "run script cli2json.py {0}"'.format(cmd)
    p = subprocess.Popen([shellCmd], shell=True,
        stdin = subprocess.PIPE,
        stdout = subprocess.PIPE,
        stderr = subprocess.PIPE)
    (stdoutdata, stderrdata) = p.communicate()
    try:
        jsonData = json.loads(stdoutdata)
    except:
        jsonData = {JSONERROR:'error processing command {0}'.format(cmd)}
    return jsonData

#example calling an EXOS show command

jsonData = exos2json('show ports 1 statistics no-refresh')
print json.dumps(jsonData, sort_keys=True, indent=2)

[
  {
    "CLIoutput": "Port      Link       Tx Pkt     Tx Byte      Rx Pkt     Rx Byte      Rx Pkt      Rx Pkt      Tx Pkt      Tx Pkt\n          State       Count       Count       Count       Count       Bcast       Mcast       Bcast       Mcast\n========= ===== =========== =========== =========== =========== =========== =========== =========== ===========\n1         A             147       26426         147       26036           0          74           0          72\n========= ===== =========== =========== =========== =========== =========== =========== =========== ===========\n          > in Port indicates Port Display Name truncated past 8 characters\n          > in Count indicates value exceeds column width. Use 'wide' option or '0' to clear.\n          Link State: A-Active, R-Ready, NP-Port Not Present, L-Loopback\n"
  },
  {
    "show_ports_stats": {
      "dot1dTpPortInDiscards": 0,
      "dot1dTpPortInFrames": 147,
      "dot1dTpPortMaxInfo": 1500,
      "dot1dTpPortOutFrames": 147,
      "linkState": 1,
      "port": 1,
      "portList": 1,
      "portNoSnmp": 1,
      "rxBcast": 0,
      "rxByteCnt": 26036,
      "rxMcast": 74,
      "rxPktCnt": 147,
      "txBcast": 0,
      "txByteCnt": 26426,
      "txMcast": 72,
      "txPktCnt": 147
    },
    "status": "SUCCESS"
  }
]
(Edited)
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb
Hi Dave,

thanks a lot for your great support! Since my partner already worked on the XML-capturing, we're going for the way to connect to the switch via telnet and run the script periodically.

But I have another question: The XML-output contains a lot of information and we don't know what every piece of the information means. Is there anything like a reference guide where we can look up what information is contained in the XML-output of specific commands?

Thanks again :)
Mirco
Photo of Dave Hammers

Dave Hammers, Dir SW Engineering

  • 3,792 Points 3k badge 2x thumb
Hi Mirco,
We don't have anything like a field by field description.
The relationship of what a CLI displays and the raw data that went into it is easier to see with:

run script cli2json.py -d <any EXOS show command>.

This built in script gives the CLI output and any data that was used to create the CLI for a visual mapping.
There are hundreds of CLI commands, if you have a few that you are using, I could ask the engineers for an explanation of the fields that you have questions about.
DaveH
Photo of Mirco

Mirco

  • 162 Points 100 badge 2x thumb
Hi Dave, thanks again for your support! Back from vacation ;) Actually, we're interested in the four following commands:

'show msrp'
'show mvrp'
'show msrp streams'
'show network-clock gptp'

The majority of information given through these commands are clear, but it would be great to have a documentation/explanation about those outputs, just to be sure we're interpreting it correctly!

Thanks!
Mirco
Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 13,676 Points 10k badge 2x thumb
yep, I should have been more precise in the doc, the list of commands is a dictionary. You can see the function tried to execute your argument as a dictionary, you have as many Error as letters in your command (space included).
Photo of Tim Black

Tim Black

  • 480 Points 250 badge 2x thumb
As described in my thread over here, I'm interested in learning which EXOS API is currently the most feature rich, and the most extensible for remote control and introspection of switch configuration and information. I'm using these switches in various topologies for AVB media transport, so in addition to controlling things like enabling/disabling ports, inline-power and vlans, like Mirco here, I will also be wanting to show msrp, mvrp, streams, and gptp info. 

I believe today the answer is to either:
1. use jsonrpc.cli() and parse the json response in the remote python application.
2. do same with restapi.

Stephane, et al, is this correct?

Jsonrpc will be my choice, since json is already a very pythonic structure, and will integrate more rapidly in my python-based system. However, other than this interface preference an application environment might have, would there be any reason to choose your REST API, or are they both equally capable in terms of switch control and introspection bc they are both just wrappers around the CLI? Thanks.
Photo of Dave Hammers

Dave Hammers, Dir SW Engineering

  • 3,802 Points 3k badge 2x thumb
The REST APIs are based on OpenConfig data models. The EXOS implementation is available starting with 22.3 and continues to include more models over time. The REST models are published on github so as more models are supported, they can be downloaded to previous EXOS releases.

That said, the REST models are not a complete set of data elements to manage a switch. If you wanted some data elements outside of the defined data models the JSONRPC would help.

JSONRPC is CLI-centric therefore proprietary to EXOS. Because everything can be done via CLI, JSONRPC offers the same configuration/mgmt coverage as SSH/Telnet/Console.

The difference for data retrieval 'show' commands is JSONRPC returns the internal data structures used to format the CLI display. The data structures are proprietary by quite often self explanatory. If you're developing in Python, it beats screen scraping.

The other method offered by JSONRPC is the ability to push a Python script down to a switch and run it without having to first transfer the script to the switch.

REST API - based on standardized OpenConfig models but not complete
JSONRPC - proprietary but complete EXOS mgmt 
Photo of Tim Black

Tim Black

  • 480 Points 250 badge 2x thumb
Thanks Dave. I see your name as the last person to update the version of jsonrpc.py in the EXOS_Apps project (back in March), so I will ask you my next question directly:

I also see an implementation for the JsonRPC class required to use this API defined in the rmtscript.py module in the ExtremeScripting project, which was modified much more recently (3 months ago). I'm confused by these multiple implementations - seems like one should be using the other, since they're both in ExtremeNetworks' repos. Which should I use, to maximize my chances of being able to pull improvements moving forward, and/or be able to contribute to?

I was going to just use the one in the EXOS_Apps project, since this is the one that Stephane Grosjean pointed me at in my other thread. But that one apparently isn't the most up-to-date.

Any advice would be appreciated, thanks.
     
Photo of Dave Hammers

Dave Hammers, Dir SW Engineering

  • 3,802 Points 3k badge 2x thumb
I'd recommend using https://github.com/extremenetworks/EXOS_Apps/tree/master/JSONRPC
It has support for authentication cookies so each JSONRPC transaction doesn't re-authenticate on the switch.
It also has HTTP/HTTPS support which tries HTTPS first before trying HTTP.

I had forgotten about the version posted in ExtremeScripting but I see someone has contributed to that version.which is why the date is recent.
Photo of Tim Black

Tim Black

  • 480 Points 250 badge 2x thumb
Thanks Dave. Do you or anyone have any general insights into the purpose and distinction between all these repos that appear to overlap a lot in that they all have examples of extreme scripts: ExtremeScripting, EXOS_Apps, exoslib?

I'm cloning what I need to our git server, and would rather not just clone everything willy nilly. Was hoping someone could point out the rhyme and reason for the code organization there. Thanks.
Photo of Tim Black

Tim Black

  • 480 Points 250 badge 2x thumb
I ended up just copying the jsonrpc.py file from EXOS_Apps for now to our internal repo, since that was all that was required. It works great! We'll look into using the entire repos when our needs get more complicated. Thanks.