Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions docs/apis/imaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Imaging features in Windows AI Foundry support the following capabilities:
- [**Image Super Resolution**](#what-can-i-do-with-image-super-resolution): scaling and sharpening an image.
- [**Image Description**](#what-can-i-do-with-image-description): generating text that describes an image.
- [**Image Segmentation**](#what-can-i-do-with-image-segmentation): identifying objects within an image.
- [**Image Foreground Extractor**](#what-can-i-do-with-object-erase): foreground/background segmentation on an input image
- [**Object Erase**](#what-can-i-do-with-object-erase): removing objects from an image.

For **API details**, see [API ref for AI imaging features](/windows/windows-app-sdk/api/winrt/microsoft.windows.ai.imaging).
Expand Down Expand Up @@ -353,6 +354,64 @@ ImageObjectExtractorHint hint(
);
```

## What can I do with Image Foreground Extractor?

Use ImageForegroundExtractor to segment the foreground or background of an input image and enable features such as background removal and sticker generation.

The returned mask is in grayscale-8 format with the pixels of the foreground mask having a value of 255 and the pixels of the background having a value of 0.

### Generating a Mask from a SoftwareBitmap

1. Call GetReadyState() and wait for EnsureReadyAsync to complete successfully to confirm that the ImageForegroundExtractor object is ready.
2. After the model is ready, call CreateAsync to instantiate an ImageForegroundExtractor object.
3. Call ImageForegroundExtractor.GetMaskFromSoftwareBitmap() with the input image to generate the foreground mask.

```csharp
using Microsoft.Windows.AI.Imaging;
using Microsoft.Windows.AI;

if (ImageForegroundExtractor::GetReadyState() == AIFeatureReadyState.EnsureNeeded)
{
var result = await ImageObjectRemover.EnsureReadyAsync();
if (result.Status != PackageDeploymentStatus.CompletedSuccess)
{
throw result.ExtendedError;
}
}

var model = await ImageForegroundExtractor.CreateAsync();

// Insert your own softwareBitmap here.
var foregroundMask = model.GetMaskFromSoftwareBitmap(softwareBitmap);
```

```cpp
#include <winrt/Microsoft.Graphics.Imaging.h>
#include <winrt/Microsoft.Windows.AI.Imaging.h>
#include <winrt/Windows.Graphics.Imaging.h>
#include <winrt/Windows.Foundation.h>
using namespace winrt::Microsoft::Graphics::Imaging;
using namespace winrt::Microsoft::Windows::AI.Imaging;
using namespace winrt::Windows::Graphics::Imaging;
using namespace winrt::Windows::Foundation;

if (ImageForegroundExtractor::GetReadyState() == AIFeatureReadyState::NotReady)
{
auto loadResult = ImageForegroundExtractor::EnsureReadyAsync().get();

if (loadResult.Status() != AIFeatureReadyResultState::Success)
{
throw winrt::hresult_error(loadResult.ExtendedError());
}
}

auto model = co_await ImageForegroundExtractor::CreateAsync();

// Insert your own softwareBitmap here.
auto foregroundMask = model.GetMaskFromSoftwareBitmap(softwareBitmap);

```

## What can I do with Object Erase?

Object Erase can be used to remove objects from images. The model takes both an image and a greyscale mask indicating the object to be removed, erases the masked area from the image, and replaces the erased area with the image background.
Expand Down