Skip to content
This repository has been archived by the owner on Jun 15, 2021. It is now read-only.

WIP Produce the simplest-possible getting started guide for the Bisq HTTP API #54

Closed
wants to merge 16 commits into from
Closed
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
252 changes: 252 additions & 0 deletions http-api-monitor-offers.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
= Monitoring Bisq offers with Http API

:sectlinks:
:sectanchors:

This guide walks you through the process of creating a bot that monitors available offers.

== What you'll build

You'll build a NodeJS-based script that connects to Bisq over an HTTP API to get offers and market prices and displays "interesting" offers on the console if any is found.

== What you’ll need

* About 15 minutes
* A favorite text editor or IDE
* NodeJS 6+
* Docker (to run from an image) or Git and Maven (to build from source)

[NOTE]
Bisq HTTP API is currently incubating. This means that things may be a bit rough around the edges
and your feedback (see the link:#next-steps[next steps] section for details).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Right about here we should do an INFO admonition that lets the user know that the Bisq HTTP API is currently incubating, and that this means things may be a bit rough around the edges, that their feedback is extra appreciated, along with pointers about how to provide that feedback. You may just direct them to the "next steps" section at bottom where they can find out more about providing feedback (see my related comment about 'next steps' below).

== Run the API

There are two alternative ways you can run an instance of the Bisq HTTP API, either from a Docker image or from source.

=== Run the API using Docker

The easiest way to run the API in headless mode is by using a Docker image:
Copy link
Contributor

@m52go m52go Jun 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could 'headless' be clarified here? Knowing virtually nothing about how this API works, my initial impression was that the API would query my local Bisq client for data, and that 'headless' simply meant 'no GUI' but that's clearly not the case.

Basically I wasn't sure if the API needed my Bisq client to be running in the background.

A quick 1 or 2 lines about what these images actually do / how the API works might help?

Also, how does one know when the Docker image is done setting up? Is it when the Bisq ASCII art shows? I watched the terminal for a while until I realized that what I was seeing was actually streaming data, and API calls were already working.


docker run -it --rm -p 8080:8080 -e BISQ_API_HOST=0.0.0.0 bisq/api

=== Run the API from source

For more hard-core developers that want to run from source:

git clone https://github.com/mrosseel/bisq-api
cd bisq-api
mvn compile exec:java \
-Dexec.mainClass="network.bisq.api.app.ApiMain" \
-Dexec.args="--appName=http-api-monitor-offers"

[NOTE]
Since the API is in incubating phase we suggest to run it against different database directory then the default one.
To do that, use `--appName` parameter. You can set it to whatever you like, just keep it different from `Bisq`.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Right about here we should do an INFO admonition that explains why they're having to run a separate instance of Bisq to experiment with the HTTP API. It's going to be confusing as hell for users who have a perfectly good Bisq client already up and running locally to spin up a different, unrelated instance, unless they have that explained to them. I'd just explain it very clearly in the context of this being an "incubating" project, and that when the the project comes out of incubation, this functionality will be integrated directly into the official Bisq client you run locally. In the meantime, you need to spin up a separate instance to experiment with (and provide feedback on!) this incubating project.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

=== Verify the API is running

In both cases the API should be running on port 8080. You can verify that by executing:

curl http://localhost:8080/api/v1/version

You should receive a response like the following:

[source,json]
----
{
"application": "0.6.7",
"network": 1,
"p2PMessage": 10,
"localDB": 1,
"tradeProtocol": 1
}
----

[NOTE]
Complete interactive API documentation is available at http://localhost:8080/swagger.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd make this stand out a bit more. I don't know if it was just me, but I glazed over it the first few times I saw it. (I also wasn't aware of Swagger, so I just thought it was a cool pathname).

Maybe take it out of a NOTE and make it something like "*Complete API documentation is available at http://localhost:8080/swagger*. It's interactive, so you can play around with it right in your browser."


== API overview

First we will look at the endpoints we need to fetch the data from and how they responses look like.

=== List available offers

Open offers are available at http://localhost:8080/api/v1/offers.

The response should look like this:

[source,json]
----
{
"offers": [OfferDetail],
"total": integer
}
----

Where `offers` is an array with individual offers and `total` is number of all offers. The model for each individual `OfferDetail` is defined as follows:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure there's a lot of value in including the whole model here. As a reader, I feel like I'm supposed to do something with this, when in fact it's just for informational purposes. As an alternative, I'd deep-link the user to the swagger API docs that show this same information, and possibly do it in an INFO admonition so that the reader understands this is "just if they care / want to go deeper", and that it's not part of the critical path of the step-by-step guide.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is problematic cause the link looks like http://localhost:8080/swagger#/offers/find3. That find3 might change into something else if we modify the resource in future by adding/removing GET endpoints.


[source,json]
----
{
acceptedBankIds [string]
acceptedCountryCodes [string]
amount integer($int64)
arbitratorNodeAddresses [string]
bankId string
baseCurrencyCode string
blockHeightAtOfferCreation integer($int64)
buyerSecurityDeposit integer($int64)
counterCurrencyCode string
countryCode string
currencyCode string
date string($date-time)
direction string Enum: [ BUY, SELL ]
hashOfChallenge string
id string
isCurrencyForMakerFeeBtc boolean
isPrivateOffer boolean
lowerClosePrice integer($int64)
makerFee integer($int64)
makerPaymentAccountId string
marketPriceMargin number($double)
maxTradeLimit integer($int64)
maxTradePeriod integer($int64)
minAmount integer($int64)
offerFeePaymentTxId string
ownerNodeAddress string
paymentMethodId string
price integer($int64)
protocolVersion integer($int32)
sellerSecurityDeposit integer($int64)
state string Enum: [ UNKNOWN, OFFER_FEE_PAID, AVAILABLE, NOT_AVAILABLE, REMOVED, MAKER_OFFLINE ]
txFee integer($int64)
upperClosePrice integer($int64)
useAutoClose boolean
useMarketBasedPrice boolean
useReOpenAfterAutoClose boolean
versionNr string
}
----

Now let's assume that we want to buy bitcoin, and let's define "interesting" offers as those that:

. sell bitcoin at a discount 1% or more under the current market price, and
. accept payment in Euros to a Polish SEPA account

First we need to filter those offers using following static criteria:

[source,json]
----
{
baseCurrencyCode: 'BTC',
counterCurrencyCode: 'EUR',
direction: 'SELL',
paymentMethodId: 'SEPA'
}
----
Copy link
Member Author

@cbeams cbeams May 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • This result and the "we need to filter those offers" language above again make me as a reader feel like I'm supposed to do something with this information. There is actually no "step" here, it's just for informational / context purposes, and that's fine, but it should be made more clear. For example, you can say something like:

We'll need to filter our offers using the static criteria you see below:

[json snippet]

In the sections that follow, we'll do just that with a combination of data returned from Bisq HTTP API endpoint calls and programmatic filtering.

(Very quickly written, but you get the idea)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List available offers and Get the market price parts are just describing how the API looks like.
I wanted to first give API overview and then dive into the javascript.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've introduced API overview section and those List available offers and Get the market price are now subsections so it should be clear for the user that they don't need to do anything there.


Next we need to filter those offers by price. There are two types of offers: _market-based price offers_ and _fixed price offers_. They are distinguished by the `useMarketBasedPrice` attribute. In case of market-based price offers the filtering criteria is easy: the `marketPriceMargin` value must be above 0.1. In case of fixed price offers we have to fetch market price of BTC in EUR, then calculate whether the price is 1% or less, and finally we must filter offers which have a price _less_ than that calculated price.

=== Get the market price

In order to get the market price of BTC in EUR, execute the following query:

curl http://localhost:8080/api/v1/currencies/prices?currencyCodes=EUR

You should receive a response like the following:

[source,json]
----
{
"prices": {
"EUR": 7035.62
}
}
----

== Write the monitoring bot code

Let's install some dependencies:

npm install lodash http-as-promised
Copy link
Contributor

@m52go m52go Jun 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be helpful to mention adding the require() statements here too.

I built the script as I read along, and ended up with errors because I didn't require() those modules. It's a super-simple thing that most people will figure out quickly, and I know it's already in the full script at the bottom, but just a quick mention along the lines of "don't forget to require these modules" or something might be nice.


In general our script will look like this:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the sections that follow, I recommend that all source code snippets:

  • get sourced from an actual, e.g. 'bisqmon.js' source file checked into the repository. Or perhaps http-api-monitor-offers.js to align with the naming of this file. We'll probably need to think more about how to structure companion sources along with these docs in a flat namespace, but it doesn't matter much at this point: what does matter is that the code below should get extracted as snippets from real source code that actually works. Asciidoctor has great tools to help here, see https://asciidoctor.org/docs/user-manual/#include-partial.

  • get labeled such that they show the name of the file they're coming from (it's fine if all three are labeled as coming from the same file. The point is that this guide needs to be an extremely literal, step-by-step process of going from zero to working code. So if there is code that the user should write, you should tell them where to write it (i.e. in a file named bisqmon.js) and in which order to write it. Add this statement, add this function, and that function, and you're done.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both done


.http-api-monitor-offers.js
[source,javascript]
----
include::http-api-monitor-offers.js[tags=flow]
----

So first 2 things to do (concurrently and asynchronously) is to fetch offers and market price of BTC from our API. Then we need to filter those offers and finally display the results.

Getting offers is a simple call that returns promise with array of offers:

.http-api-monitor-offers.js
[source,javascript]
----
include::http-api-monitor-offers.js[tags=getOffers]
----

Similarly with `getMarketPrice`:

.http-api-monitor-offers.js
[source,javascript]
----
include::http-api-monitor-offers.js[tags=getMarketPrice]
----

Now our `filterOffers` function is ready to receive an array of results from the two functions described above:

.http-api-monitor-offers.js
[source,javascript]
----
include::http-api-monitor-offers.js[tags=filterOffers]
----

This function filters offers to match our criteria. It returns matching offers and maps them to a bit simpler structure that contains as little data as needed for `notify` function. We are using `lodash` library to simplify the filtering.

The `getPriceFilter` function creates the actual filter function and looks like this:

.http-api-monitor-offers.js
[source,javascript]
----
include::http-api-monitor-offers.js[tags=getPriceFilter]
----

We are multiplying `marketPrice` by `10000` because that is the format in which the API returns the price.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Somewhere right about here, you should instruct the user to actually run node bisqmon.js (or whatever the correct invocation should be), and then show them what they should expect to see as correct output. This is the big payoff. They've hung with us for 15 minutes, reading our instructions, taking in the context, and following each step faithfully. Now they get what they came for: working code that does exactly what we said it would.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


Here is a full script.

.http-api-monitor-offers.js
[source,javascript]
----
include::http-api-monitor-offers.js[tags=**;*]
----

Now run

node http-api-monitor-offers.js

If there are any matching offers you should see something like this:

5 interesting BTC offers from Bisq
0.0625 BTC (-2%)
0.01 BTC (-2%)
0.01 BTC (-5%)
0.033 BTC (-3%)
0.02 BTC (-1.5%)
0.25 BTC (-6%)

If there are no matching offers, try fiddling with the value of `threshold`. Try setting it to -20, 0.01, etc.

== Congratulations

You are now able to monitor Bisq offers via HTTP API!

== Next steps

* Try adding a loop to keep the process repeating the search periodically (use _setInterval_)
* Join our slack at https://bisq.slack.com and leave feedback on the API and this guide
* Report https://github.com/mrosseel/bisq-api/issues[issues]
62 changes: 62 additions & 0 deletions http-api-monitor-offers.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
const _ = require('lodash');
const $http = require('http-as-promised');

const threshold = 0.1;

//tag::getPriceFilter[]
function getPriceFilter(marketPrice) {
const maxPrice = marketPrice * (1 - threshold) * 10000;
return offer => {
if (offer.useMarketBasedPrice)
return offer.marketPriceMargin >= threshold;
return offer.price < maxPrice;
}
}
//end::getPriceFilter[]

//tag::getMarketPrice[]
function getMarketPrice() {
return $http.get('http://localhost:8080/api/v1/currencies/prices?currencyCodes=EUR', {resolve: 'body', json: true})
.then(body => _.get(body, 'prices.EUR'))
}
//end::getMarketPrice[]

//tag::getOffers[]
function getOffers() {
return $http.get('http://localhost:8080/api/v1/offers', {resolve: 'body', json: true}).then(body => body.offers);
}
//end::getOffers[]

function notify(offers) {
Copy link
Contributor

@m52go m52go Jun 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from the require() statements, this notify() method was the one other item that wasn't mentioned in the text that I didn't have in my code when I ran the script for the first time.

Just FYI. I guess it's because this is structured as a bottom-up walk-through. If it were reversed (i.e., top-down approach with full script first and then important parts clarified), it would be fine to pick and choose what to cover.

if (!offers.length) {
console.log('No interesting offers found');
return;
}

const text = _.map(offers, offer => `${offer.amount / 100000000} BTC (-${_.round(offer.margin * 100, 2)}%)`).join('\n');
console.info(text);
}

//tag::filterOffers[]
function filterOffers([offers, marketPrice]) {
return _(offers)
.filter({
baseCurrencyCode: 'BTC',
counterCurrencyCode: 'EUR',
direction: 'SELL',
paymentMethodId: 'SEPA'
})
.filter(i => _.includes(i.acceptedCountryCodes, 'PL'))
.filter(getPriceFilter(marketPrice))
.map(i => _.pick(i, 'baseCurrencyCode', 'counterCurrencyCode', 'direction', 'paymentMethodId', 'id', 'useMarketBasedPrice', 'price', 'marketPriceMargin', 'amount', 'minAmont'))
.map(i => ({amount: i.amount, margin: i.useMarketBasedPrice ? i.marketPriceMargin : marketPrice / i.price}))
.value();
}
//end::filterOffers[]

//tag::flow[]
Promise.all([getOffers(), getMarketPrice()])
.then(filterOffers)
.then(notify)
.catch(e => console.error(e));
//end::flow[]
1 change: 1 addition & 0 deletions index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
* *_User Docs_*
** <<intro#, Introduction>> — What Bisq is, why it exists and how it works
** <<getting-started#, Getting Started>> — Go from zero to trading in 15 minutes
** <<http-api-monitor-offers#, Monitoring Bisq offers with HTTP API>> — Getting started with Bisq HTTP API

* *_Contributor Docs_*
** <<contributor-checklist#, New contributor checklist>>
Expand Down