Setup Assistance

[duckhead]

So, I've been asked to look into a few OSMT issues by a very pretty lady sitting next to me. I've got lots of Java, but am a Docker noob. So, I'm just trying to be able to get the environment built and running, so I can poke around at it.

I've got a few questions (and, I'm happy to update the readme.md with the answers, I've done a lot of doc.)

First, a note, Maven 3.8.6 no longer exists where you attempt to pull it from (I don't know why...) I had to update the script to 3.8.8. I've got that fixed in my local copy.

Secondly, I have attempted to get an OKTA account. And I think I was successful. But, I don't know how to get the information that you want in your file. I'd love to get a good explanation soup to nuts and I'll update the doc.

When I run docker-compose on dev-stack.yml, it looks like everything starts, but I see that MySQL is angry that I didn't supply the MYSQL* environment variables. I need to fix that, but I'm not sure what I need to do. I see some references to that in the osmt_cli.sh script, but there it gets default values, and apparently it wasn't there at the mysql container startup.

Finally, this is kind of the end of the docs. I'm not sure what is next. What are the base and build containers? The base container is just a JDK, wget, and curl? So, I use that as my dev environment? How do I do that with the sources not being in there? The build container is just Maven? How am I to use that, just to build? Again, I'm happy to update the docs.

[duckhead]

OK, I've been hacking on this some more, and I think all of the above issues are resolved.

It took me a bit to follow through the Okta documentation, and there was at least one choice I was asked to make that wasn't in the doc. I either did or did not guess correctly (I'll explain that later.)

I was able to build the solution, and actually run it using osmt_cli.sh. It starts up, and I can connect to it on localhost:8080 . That all is fine. From there, I pull up the webpage, and got the WGU page, with the spinner in the middle. But, it kept spinning. Eventually, I was taken to a Okta webpage complaining that I was apparently trying to authenticate too many times. So, I'm probably DOSing them.

I'd love to look at the logs, but I don't know where to find them. Does anyone have a pointer for me?

[JohnKallies]

You were asking about the Docker images before, base and build. Those are really only used in support of OSMT Spring app images that go to DockerHub. You can build them locally, but I never saw where that gets you much.

[duckhead]

It's all good, I like tinkering with these sorts of things. I'm still working with a fresh install to make sure I get the overall flow documented.

[duckhead]

OK, next issue (this one is cute.) I'm having some issues with my ElasticSearch container dying on me. I also notice that it's eating a lot more memory than I expect it to, like around 16GB., which is a couple orders of magnitude larger than my MySQL container. I suspect those are related, but I've not been able to prove it. docker logs provides no hints as to what is going on, but I did find this, which is curious:

{"type": "server", "timestamp": "2023-09-28T22:13:32,179Z", "level": "INFO", "component": "o.e.i.g.ConfigDatabases", "cluster.name": "docker-cluster", "node.name": "48ca258a04bf", "message": "initialized default databases [[GeoLite2-Country.mmdb, GeoLite2-City.mmdb, GeoLite2-ASN.mmdb]], config databases [[]] and watching [/usr/share/elasticsearch/config/ingest-geoip] for changes" }
{"type": "server", "timestamp": "2023-09-28T22:13:40,161Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "48ca258a04bf", "message": "updating geoip databases", "cluster.uuid": "Cz5UgcbrQmS4AcevuPUHOA", "node.id": "Yq7aemVxT4SYPdXdMx3m4w" }

It seems to be loading GeoIP data, which is not a thing I would think that OSMT would want or need.

It looks to be a known issue, and I'm not comfortable enough in Docker to implement the fix though:

elastic/elasticsearch#76586

[JohnKallies]

To confirm, OSMT should not have any concept of GeoIP data. Best guess is that ou are getting it for free from ES or Spring Data. I have seen this, and I don't really know why.

It doesn't surprise me that ES uses a lot more than MySQL. In OSMT, MySQL is the source of truth, but all reads come from ES, and we have indexed for case insensitivity in a few places. That said, I would think this would bubble up in IO more than in RAM consumption.

I use a Mac, and within Docker Desktop (the GUI for configuring the Docker service), I have allowed 12GB RAM.