QRadar App Boilerplate
I want to walk through how to write a QRadar app, specifically to collect logs from a log source that only allows the use of an API and that QRadar does not natively support. Examples at the time of writing include Duo Security and Trend Micro Apex Central, but there are potentially thousands of others you might run into.
First off, let’s check my Github repo: https://github.com/freehunter/QRadar-API-Connector-Template
Secondly, you should have a QRadar system. Best practice would be to develop and test apps in a test environment like QRadar CE, but due to the App Framework’s use of Docker, it’s a low risk to run this in production. Just be aware you may accidentally feed in some logs that you didn’t intend to, and EPS licenses definitely still apply when you’re sending logs from an app.
Lastly, make sure you have the QRadar App Editor downloaded and installed. You can use the official SDK as well, but I don’t recommend it for simple applications like this. There’s nothing wrong with the SDK, I just believe it’s just cumbersome unless you need your own editor, CI/CD, git support, Node.js support, etc. We don’t need it for our purposes here.
Open your QRadar system, navigate to the Admin section, and click Develop Applications. Select New App and then Config Page Sample. Fill in the rest of the details as you see fit and click Install.
After the new app and editor are installed, refresh the page and click the new tab to develop your app. Here’s where we’re going to get into it. Look at the file browser to the left. There are two sections you’ll really need to pay attention to: views.py, and templates/index.html. Views.py is where your server-side code goes (almost all of your code). Index.html is what will show up when you click the button in the Admin section. I’m not going to get into that here, but maybe in future posts. For now, index.html is where you’ll want to plug any client-side code.
Open views.py and remove all the code. Plug in all the code from the boilerplate I linked to above, from boilerplate.py.
Hopefully the import section is explanatory. I’ve pre-filled a few things that I’ve used commonly in the past, even if they’re not used in this particular code. Feel free to remove any unneeded imports.
The second section is where things get interesting. For these apps, you need them to run constantly in the background, starting immediately upon deploying the code.
@app.before_first_request
def activate_job():
def run_job():
global poll_time
console_ip = qpylib.get_console_address()
while True:
getLogs()
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.sendto(json.dumps('Heartbeat'), (console_ip, 514))
time.sleep(poll_time/1000)
thread = threading.Thread(target=run_job)
thread.start()
The before_first_request makes sure this code is run as soon as the deploy is finished, and the while True makes sure it runs forever. Then I run the getLogs() function which we will explore in a moment, followed by sending a heartbeat to QRadar so we can set up rules to watch if the app has crashed or failed (remind me to cover this in a future post).
I want to take a second to cover the qpylib line. In your file browser to the left, open qpylib/qpylib.py. The developers have helpfully created a library that covers a lot of common functions, like get_console_address(). Feel free to explore this and use these functions in your own code. For now we’re really just using this one, to get the console IP address.
I’m skipping the section about the admin screen because we’re not covering client-side code here. We simply don’t need it for an API connector.
def getLogs():
global poll_time
console_ip = qpylib.get_console_address()
# Get logs starting from x seconds ago through right now
min_time= str(int(round(time.time() * 1000)) - poll_time)
max_time = str(int(round(time.time() * 1000)))
payload = {'mintime': min_time, 'maxtime': max_time}
# Make the request
r = requests.get('https://api.example.com', params=payload)
# Change the JSON string into a JSON object
jsonObject = json.loads(r.text)
This is pretty simple stuff, and mostly taken right from the Requests library documentation. I have a global variable for poll_time to set how often the app checks for new logs, then we’re grabbing the console IP address.
Very often APIs will want you to specify a min_time and a max_time to make sure you’re grabbing the most relevant logs. That piece will have to be up to you; I’ve shown one example but this will vary quite heavily.
Lastly we make the request and format the response. This is the part pulled right from the Requests library and couldn’t be more simple. Check out the duo.py file to see how this can get much more complicated. As one example, Duo needs a weird authentication mechanism and that needs to be included in the headers. So before I call requests.get I needed to call a function I named sign() to sign the request before sending it.
if "response" in jsonObject:
resp = jsonObject["response"]
# Convert the logs to syslog and feed to QRadar
for MESSAGE in resp['authlogs']:
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.sendto(json.dumps(MESSAGE), (console_ip, 514))
Last section of the boilerplate checks to make sure the response is what we expected (hint: “response” likely won’t be what you’re looking for) and loops through the JSON response to feed each individual log to QRadar in one single message.
The hardest part will be getting the API call done properly. For this I use the excellent app Postman, which lets you explore APIs interactively. Once you have something that works in Postman, on the right side of the screen there is a link that says Code, which you can click to get pre-written Python/Requests code to plug in to your app. Use it, it’s fantastic.
Deploying
To deploy this app from here, click Actions -> Deploy App – > In Development Mode. Click yes to both prompts and be patient. If you click Live Mode, you lose the ability to edit the code anymore. Don’t do that unless you’re sure you’re done with it.
If you wrote this on a dev/lab system, you’ll want to deploy in Live Mode and then use the content management tool from the command line to extract the app so you can install it on your production system. Exporting the app in development mode leaves the App Editor window in place on the other system (more info here).
What Next?
After walking through this boilerplate, hopefully you have what you need to start writing API connectors. If you want practice (hopefully in a test/lab system), try writing an app against the JSON examples site. When I was practicing I wrote an app against one of my hobby projects, so every time I wrote a new blog post it would get fed into QRadar. Silly? Yes. Educational? Absolutely.