Oracle REST Data Services 22.3 was released a couple of weeks ago, and it is now available on Oracle Autonomous Database as well! This release has a slew of new REST APIs for Oracle Transactional Event Queueing (or TxEventQ). If you have not heard of it, TxEventQ is essentially a new, faster implementation of Oracle Advanced Queueing which has been in the database for around twenty years.
Many of these new REST APIs are very simliar to the Kafka REST APIs, since TxEventQ provides Kafka compatibility as one of its features.
In this post, I want to show you how to use a few of the APIs, and then I’ll give you an idea of what kinds of APIs are available and where to find more information.
The first thing to do is the grab an Autonomous Database instance. It’s available in the Always Free tier, so you can try this for free! If you are not familiar with creating one, and accessing SQL and so on – check out this free LiveLab for details.
Make sure you get a 21c database – you may need to toggle the “Always Free” option to see 21c. The APIs described in this post are supported in Oracle Database 21c (and later).
When you get into your SQL worksheet, grab the URL from the browser, it will be something like this:
https://xyzabc-red1.adb.us-phoenix-1.oraclecloudapps.com/ords/admin/_sdw/?nav=worksheetNow, chop off the end of the URL and replace it with the base URL for the TxEventQ REST APIs, and save that in an environment variable to save us some typing!
export ADDR="https://xyzabc-red1.adb.us-phoenix-1.oraclecloudapps.com/ords/admin/_/db-api/stable/database/teq"And let’s create another environment variable with the authentication details. You can encode them using base64 like this, assuming your userid is admin and your passsword is your_password:
$ echo -n "admin:your_password" | base64
YWRtaW46eW91cl9wYXNzd29yZA==Then we can use that value to create the authentication header details:
export AUTH="Authorization: Basic YWRtaW46eW91cl9wYXNzd29yZA=="Great, that will save us from typing those each time!
Create a topic
Let’s start with by creating a topic. We are going to need to know the database name for this – you can find that by running this query in your SQL worksheet:
select sys_context('userenv','db_name') from dualYou’ll need to put that database name into the URL below after “clusters” and before “topics”, in this example my database name is “XYZABC_RED1“:
curl -X POST -H "$AUTH" -H "Content-Type: application/json" -d '{"topic_name": "mark1", "partitions_count": "6"}' "$ADDR/clusters/XYZABC_RED1/topics/"In the body we specified the name of the topic (“mark1” in this case) and how many parititions we want the topic to have. When you run this request, you’ll see output something like this:
{"name":"MARK1","partitions":[{"partition":0,"leader":1,"replicas":1}]}It created our topic for us!
List topics
Let’s list the topics now, try this request:
curl -X GET -H "$AUTH" "$ADDR/topics/"The output will be a JSON list of topic names, like this. You might want to create a few more to make it more interesting!:
["MARK1"]Get a topic
We can also get details of a single topic like this, the topic name is in the last part of the URL:
curl -X GET -H "$AUTH" "$ADDR/topics/mark1/"The output looks like this:
{"name":"MARK1","partitions":[{"partition":0,"leader":1,"replicas":1}]}Create a consumer group
Now let’s create a consumer group, here’s the request, notice the topic name is in the body, and the name of the consumer group is the last part of the URL (“sub1” in this case):
curl -X POST -H "$AUTH" \
-H "Content-Type: application/json" \
-d '{"topic_name": "mark1"}' \
"$ADDR/clusters/XYZABC_RED1/consumer-groups/sub1/"The output from this is empty (unless you specify verbose, or get an error), but we can easily check the result in the database by running this query:
select * from user_queue_subscribersPublish messages
Ok, I think we’re ready to publish a message! Here’s the request:
curl -X POST -H "$AUTH" \
-H "Content-Type: application/json" \
-d '{"records": [{"key":1,"value":"bob"}]}' \
"$ADDR/topics/mark1/"The output will look something like this:
{"Offsets":[{"partition":0,"offset":0}]}You can put mutliple records in the body to send put more than one message on the topic.
Consume messages
Now, let’s consume the messages off that topic with our consumer sub1. Here’s the request, notice the topic name is in the body, and the soncumer name is in the URL after “consumers”:
curl -X GET -H "$AUTH" \
-H "Content-Type: application/json" \
-d '{ "partitions": [ { "topic": "mark1", "partition": 0 } ] }' \
"$ADDR/consumers/sub1/instances/1/records"The output from this one (not surprisingly) looks like this:
[{"topic":"MARK1","key":"1","value":"bob","partition":0,"offset":0}]Great, hopefully that gives you a feel for how these REST APIs for TxEventQ work!
But wait, there’s more!
Of course there are a lot more APIs available than the few I have shown you so far. They all follow a fairly similar pattern, let’s take a look at a list of what’s available:
Topics APIs
- Create topic, optionally with partition count
- List topics
- Get a topic
- Create a consumer group
- Publish message(s)
- List topics in a specific cluster
- Get a topic in a specific cluster
- Delete a topic
Partitions APIs
- List paritions in a topic
- Get details of one partition in a topic
- Get partition message offsets
- List partitions in a topic in a cluster
- Get details of one partition in a topic in a cluster
- List consumer lags for a partition
- Publish message(s) in a particular partition
Consumer Group APIs
- List consumer groups
- Details of one consumer group
- Get consumer group lag summary
- Get consumer group lags for all partitions
- Get consumer group lags for a given partition
- Delete consumer groupworking
- Create consumer group
Consumer APIs
- Create consumer instance on consumer group
- Delete consumer instance on consumer group
- List topics that a consumer is subscribed to
- Subscribe to topics
- Unsubscribe from topics
- Send message to given partition
Move Offset APIs
- Fetch messages
- Fetch messages from offset in specified partition
- Move to beginning of partition
- Move to end of partition
- Get offsets in specific topics and paritions
- Commit (ser) offsetes in specific partitions
Where to find more information
You can find more informaiton about the TxEventQ REST APIs in the documentation, here: https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.3/orrst/api-oracle-transactional-event-queues.html
Or if you prefer, you can open the OpenAPI specification on your database instance. The URL will be something like this, and you can search the output for “teq” to find the APIs:
https://xyzabc-red1.adb.us-phoenix-1.oraclecloudapps.com/ords/admin/_/db-api/stable/metadata-catalog/openapi.jsonI hope you enjoyed this quick introduction to the new REST APIs for Transactional Event Queueing! Of course, this is available in any Oracle database, not just Autonomous Database. If you want to use Oracle REST Data Services with your own database, you might find this post about installing a standalone version interesting too!
RedStack 