Delete assets from sites if there is no deviceid

[98231jssa]

Hi,
I'm managing an instance of nexpose with about 100K assets, split in quite some groups, and I have frequently the need to delete/add/tag assets in the various sites so I do it via the ruby API.

Take this example code:

nsc = Connection.new(NexposeIP, NexposeUN,NexposePW)
[...]
            if not assetid.nil? # Is a device
                $log.debug "Deleting device #{asset.hostip} - ID #{assetid} from site #{asset.sitename}"
                nsc.delete_device(assetid)
            else # is only an asset
                $log.debug "Deleting asset #{asset.hostip} from site #{asset.sitename} - Current site is #{site.name}"
                site.remove_asset(asset.hostip)
            end
[...]
site.save(nsc)

f I have an assetID the device gets always deleted properly, but the remove_asset function doesn't work as expected when the asset doesn't have an ID.
The behaviour I observe is that even if I issue a remove_asset on the site and save it, the site definition still contains the asset (in the site.assets array).
My biggest problem is that when I delete an asset from a site I must be sure it doesn't get scanned in the next discovery scan.

Is my code wrong or is it a bug?
You can cross-related this with internal support case 00122399 for more details.

[asalazar-r7]

By any chance is your site an IP range and the asset you delete in that range? for example
10.0.0.0/24 is my range and i tell the API to delete 10.0.0.9. Does the site get converted to the following range?

10.0.0.1 - 10.0.0.8  
10.0.0.10 - 10.0.0.254

[gschneider-r7]

Without looking, I suspect the logic for removing an IP doesn't check in ranges and thus is a no-op if the IP is part of a range.

[98231jssa]

Hello, I run some tests and confirmed the suspicion of @gschneider-r7:
I have test site defined as:

• 192.168.0.100-192.168.0.105
• 172.16.0.1-172.16.0.3
• 172.16.0.5

I tried to delete 4 IPs

• 192.168.0.3 not_existing
• 172.16.0.2 within_range
• 172.16.0.5 single_ip

From the logs:

Deleting asset 172.16.0.5 from site Test-DELETE
Deleting asset 172.16.0.2 from site Test-DELETE
Cannot delete device 192.168.0.3 as it doesn't exist in site 155
Deleting asset 192.168.0.103 from site Test-DELETE

Effective result is that only 172.16.0.5 (the IP NOT within range) got deleted.

Do you have a timeline for when this bug could be fixed?

Cheers!

[sgreen-r7]

@98231jssa We're working on it as we speak - aiming to have something ready by the end of the week.

[98231jssa]

@sgreen-r7 @asalazar-r7 thanks!

[erran-r7]

I started looking into this a bit. @asalazar-r7 @sgreen-r7 here's my work so far: master...erran:bug/148-allow-the-deletion-of-assets-in-ranges I've added new abilities to test whether an asset/IP is included (by string). We should be able to leverage this in the remove_* methods.

[erran-r7]

I've added more changes and the tests that I wrote to reproduce the issue here are now passing. @sgreen-r7 Can you check out my changes?

[erran-r7]

@98231jssa I've been discussing this internally since I felt a bit uncomfortable with the fix I wrote this morning.

One solution we came up with using exclusions instead. Asset exclusion is a feature we created to remove the need of removing a single asset from a site (created via a range). What that means for you is instead of calling Site#remove_asset you could call Site#exclude_asset which would leave the range as is and add it to the site's asset exclusion list. These assets will not be scanned.

If you go through my other solution you can see that we need to delete each range and split it into subsets which are then converted back into ranges.

[sgreen-r7]

@98231jssa - when you get a chance, can you let us know if @erran-r7 suggested solution would work? Using Site#exclude_asset.

The holistic fix we're trying to put in place has proved to not be as straight forward as I originally estimated. Either way, we're still working on updating Site#remove_asset in a way that will fix it in the best way possible.

[98231jssa]

Hi, I'm back and can run all the tests required.
Shall I update to 1.0 before checking again whether the Site#remove_asset works as expected or the change didn't make it for release?

[sgreen-r7]

@98231jssa The 1.0 gem release is tied to the last version of nexpose and is not backwards compatible. So if you took that nexpose update (last week) then trying the 1.0 version of the gem is a good idea. A lot changed for that update so if i'm remembering correctly - it should be fixed in a different way.

Give 1.0 a try if you have an updated Nexpose console, otherwise - we can try that branch which has the fix for the 0.9.x version of them gem. Let me know if you need some info on working off a branch of the gem.

[sgreen-r7]

@98231jssa - anything new to report?

[98231jssa]

At the moment I only have the production environment to work on, where if I update to v1.0 I break existing scripts, so I cannot test it.
I must find time to learn how to update the specific gem with the fix and schedule tests, but I also have other priorities...
pointers on how to import the right branch of 0.9.x and revert back in case of something is broken are appreciated.

Cheers and thanks again