Can we talk about creating a scanless standard API?

[zenware]

Sorry I'm on my phone so this will be terse, hopefully I have some time to update later... But something really basic, such as what I saw when I looked through the code of the other scanners. Either http to not add dependencies or maybe zmq to add speed. But effectively an easy to deploy server that enabled public users to scan IP addresses. Such that you can set it up with a docker container, a GitHub clone, or an Ansible playbook. It would be nice if there were many more publicly accessible servers to choose from or to distribute work to and I think the best route to that end goal is to write an easily deployable server. Which I'm happy to do if you think it's a good idea as well.

[vesche]

If I'm understanding this correctly, you're saying that you'd like to create an easy to deploy/distribute public port scan web service? And that could also easily (or automatically) integrate with scanless? I think that's awesome, and I support it. I think it should probably live in a separate project/repo, we could call it scanless-server or something. I'm all for docker containers. This isn't something I had thought of, but I'm interested.

I'm curious what would be the best way to integrate those servers into this project. Probably just create a scanners/scanless-server.py module and then keep a list of healthy/alive servers.

[zenware]

That's exactly my thought. Same plan for the module as well. I'd like your input however, I took a look at only one of the implemented modules so far and it had a ridiculous API.
Visibly Absurd API:

BASE_URL = 'http://ping.eu'
SCAN_LOC = '/action.php?atype=5'

Since you have some experience with this, can you give some pointers as to what an API for this type of a service might have? As well as what offerings it might support? With the existing ones, you kind of have to make-do with what's available and they may be using different tools and scanning strategies under the hood. I'm wondering if there's any feature, in particular, you sorely missed? (compared to having full control over your tools)

I'd personally like to manually specify which ports I'm scanning against, truthfully I'd have a personal default to something like nmap top 10, but then I also really like scanning for SNMP which is a UDP service and it appears that these are all TCP-only scanners?

Additionally, I think it would be funny if it were implemented as serverless functions of some sort, though I'm not sure that's as feasible as building a nice isolated server container and just hosting a bunch of em.

[zenware]

I was messing around this evening and now I have a local server written in Python using hug and multiprocessing.
I haven't had or spent a lot of time on it so obviously, it's atrocious.
Queries like this: http://localhost:8000/scan?target=8.8.8.8&ports=21,22,23,80,53,8000
Outputs like this:

{"target": "localhost", "ports": ["21", "22", "23", "80", "53"], "results": [false, false, false, false, false], "took": 0.02}

I tried to write a TCP Connect scanning thing myself and multithread it as well, but I've reconsidered.

Now I'm thinking shell out to NMAP with something like:

nmap -sS --open -oG - <target> -p<csv_port_list>

and processing the returned output

Any suggestions on what that API endpoint should actually look like?
Should requests actually come in a POST body?

[zenware]

sudo nmap -sS --open -oG - <target> -p<csv_port_list> | cut -f2 | grep "Ports:" | cut -c7- | cut -d, -f1 | cut -d/ -f1 | xargs echo because I'm lazy, this actually does most of the work in the shell, and returns a space separated list of open ports. Will make something nicer /w regex captures later.

[vesche]

Sorry I went dark on this. The API you outlined looks great, I'd be curious to see the python/hug code. Could you make a repo? I'm concerned about security when it comes to providing use input fields that will directly interact with the shell. For instance instead of target 8.8.8.8 I could put 8.8.8.8;nc -lp 1337. I'm not a big web developer, but that's just something that comes to mind. Although it really pigeonholes the capabilities, I'd like to at least start with a common set of ports it would scan. Something like nmap --top-ports 20.

[zenware]

Yeah, I was thinking it would default /w nmap top 10, but also UNIX PERMISSIONS. obviously the process runs on a locked down account on a locked down shell, that can only trigger the nmap process, AND input is sanitized (I know this is hard in the general case but I have thought of a fairly simple solution, can we DNS resolve?

[zenware]

Although...

Yes, I'll make it a repo and share it with you, it's currently non-functional just a request receiver. I think I actually want to figure out how to do a task/worker queue implementation and also my own TCP SYN/UDP scanning, partially because of the concerns you highlighted. Also because I think it will be cool to have a "100% Python Project" (Though I also want to create a PHP version, because of shared hosts everywhere.) I spent the majority of my time searching for cloud providers who don't explicitly restrict or prohibit port scanning. AWS, for example, requires you send them an application for a permit to port scan for each port scan you decide you want to run. Google cloud appears to have no such terms, neither do some VPS providers like 1and1. What I mean is, my original goal of "throw this anywhere easily" is kind of killed by providers not wanting to host this type of thing.

I think it would be cool as well to run a web service of some kind that acted as a tracker where people running their own scanless-server would update that registry with the IP of that scanless-server and it could be used to seed the list of scanless-server supporting nodes for the scanless client. I also dreamt that it would be interesting if the registry could push scan commands to the scanless-server such that a client needn't necessarily have a local copy of the list or even any copy, but just query the scanless-registry directly and it would automatically divide the workload between the list of scanless-server machines it knew about. Of course, this would all still be functional without any centralized stuff, in fact, perhaps each scanless-server could do a p2p federation sort of thing but I don't have as much experience building that architecture.

I've got a lot of ideas but not necessarily all the time in the world, let me know your thoughts if you have any.

Apologies if some of that sounded like nonsense, didn't have the time to edit. I'll publish the py/hug repo and link it here ASAP.

[zenware]

Here it is: https://github.com/zenware/scanless-server

However, as I mentioned my Python TCP scan method isn't really functional right now.

[zenware]

Wow, if you're still around and reading this I guess work and life got me distracted from side projects and I'm ready to pick up development on this again. I'll be reading through your code and mine over the weekend and hopefully standing up a real online testbed.

[vesche]

Yup, still around. I'll be interested to see what you make.

[nmmapper]

Hello Buddies, this project(scanless API) has got me really interested into. I have been in a quest to deploy some api for 8 subdomain finder

I have deployed exploit api right there and was in a quest to deploy api for the following

1. Wappalzer https://www.nmmapper.com/sys/cms-detection/wappalyzer-online/
2. Nmap at https://www.nmmapper.com/sys/networkmapper/nmap/online-port-scanning/
3. Subdomain finder https://www.nmmapper.com/sys/tools/subdomainfinder/

If this project is still something still in your mind, then i can offer the online api, I already
have API configured on the site above. Just adding /api/service/scanless/

The site is django-based
Let me know what you think.

[vesche]

I recently updated this codebase, so it would be easier now to use it in backend python code: https://github.com/vesche/scanless#library-usage

Anyone is free to use this project in any way they see fit. Change it, make money off it, even steal it and call it their own. I distributed it under the very very permissive / anti-copyright license, Unlicense: https://github.com/vesche/scanless/blob/master/UNLICENSE

Hack away friends 🤘