Way to cut drag and drop selections from the pdf?

[skyleguy]

I am trying to put something together to allow me to drag and drop on a page in my pdf and get a visible "selection box" showing while im holding the mouse down until i let go of the mouse. at that point id like to get the instance to the canvas of the page im on and cut the image from the canvas out as an image using the coordinates of the box i dragged and dropped. Is something like this already present in the library and I just have not found it yet?

If not, any ideas as to why this isnt working as expected?

My extended pdf viewer is initialized like this using a blob as the source (turned off handTool to try to allow for a typical mouse drag and drop event without the pdf moving)

<ngx-extended-pdf-viewer
  class="h-full w-full"
  [src]="selectedFileBlob"
  (pageRendered)="onPageRendered($event)"
  [handTool]="false"
></ngx-extended-pdf-viewer>

the onPageRendered method uses the pdf service to get the canvas of the page that was just rendered and sends it to another method to set up some listeners on it

onPageRendered(event: PageRenderedEvent) {
    const pageNumber = event.pageNumber;
    if (event.pageNumber) {
      this.pdfService.getPageAsCanvas(pageNumber, { scale: 1 }).then((canvas) => {
        if (canvas) {
          this.setupEventsForCanvas(canvas);
        }
      });
    }
  }

the setupEventsForCanvas method takes this canvas and adds mousedown, mouseup, and mousemove events to the canvas so that we can keep track of the rectangle and attempt to draw a box on the canvas during the duration of the drag and drop. when the mousedown event fires it should try to get the image from the canvas and set it for me to use.

private setupEventsForCanvas(canvas: HTMLCanvasElement) {
    let startX = 0;
    let startY = 0;
    let endX = 0;
    let endY = 0;
    let isDragging = false;
    const ctx = canvas.getContext('2d');
    if (ctx) {
      canvas.addEventListener('mousedown', (e) => {
        isDragging = true;
        const rect = canvas.getBoundingClientRect();
        startX = e.clientX - rect.left;
        startY = e.clientY - rect.top;
      });

      canvas.addEventListener('mousemove', (e) => {
        if (!isDragging) return;
        const rect = canvas.getBoundingClientRect();
        endX = e.clientX - rect.left;
        endY = e.clientY - rect.top;

        ctx.clearRect(0, 0, canvas.width, canvas.height);
        ctx.strokeStyle = 'red';
        ctx.lineWidth = 2;
        ctx.strokeRect(startX, startY, endX - startX, endY - startY);
      });

      canvas.addEventListener('mouseup', () => {
        isDragging = false;
        const image = ctx?.getImageData(startX, startY, endX, endY);
        if (image) {
          this.image.set(image);
        }
      });
    }
  }

my issue is that none of these events actually trigger when i click, drag and drop, mouse move, on the canvases in the pdf viewer. Any ideas how I could get something like this working? Thanks!

[stephanrauh]

Your mistake (or misunderstanding) probably is in this code snippet:

onPageRendered(event: PageRenderedEvent) {
    const pageNumber = event.pageNumber;
    if (event.pageNumber) {
      this.pdfService.getPageAsCanvas(pageNumber, { scale: 1 }).then((canvas) => {
        if (canvas) {
          this.setupEventsForCanvas(canvas);
        }
      });
    }
  }

pdfService.getPageAsCanvas() renders a fresh new canvas, specifically for you. It's independent of the canvas in the DOM tree. I haven't tried it, but I believe this code should to the trick:

private PDFViewerApplication?: IPDFViewerApplication;

  constructor(private readonly rendererFactory: RendererFactory2, notificationService: PDFNotificationService) {
    this.renderer = this.rendererFactory.createRenderer(null, null);
    effect(() => {
      this.PDFViewerApplication = notificationService.onPDFJSInitSignal();
    });
  }

  onPageRendered(event: PageRenderedEvent) {
      const pageNumber = event.pageNumber;
      if (!pageNumber) return;

      const pageDiv = document.getElementById(`page${pageNumber}`);
      if (!pageDiv) return;

      const canvas = pageDiv.querySelector('.canvasWrapper > canvas') as HTMLCanvasElement | null;
      if (!canvas) return;

      this.setupEventsForCanvas(canvas);
  }

Also note that onPageRendered can be called multiple times for the same page if the user zooms. Make sure not to define duplicate event listeners, and don't forget to unregister them in an onDestroy listener. Sometimes canvases are evicted from memory to save space, but currently, there's no event for that, so there's little you can do to prevent this particular memory hole.

Also note that you're painting on the canvas displaying the PDF page. Basically, you are overwriting it, so the text becomes illegible. Better add a custom <div> on top of the page.

[skyleguy]

thank you for the reply and notes on potential duplicate listener/listener cleanup.

I did not realize that about the getPageAsCanvas so that is a good note.

Also note that you're painting on the canvas displaying the PDF page. Basically, you are overwriting it, so the text becomes illegible. Better add a custom <div> on top of the page.

Not 100% understanding this as i have never actually used canvases before, but ill remember this if i can get any rectangles to be drawn on the canvas lol

Im a little confused from your example as it seems PDFViewerApplication and renderer are unused? I modified your code snippet (i had actually tried querying the dom like this already with no luck) to this and i see it getting instances of canvases now

onPageRendered(event: PageRenderedEvent) {
    const pageNumber = event.pageNumber;
    if (event.pageNumber) {
      if (!pageNumber) return;
      // couldnt find elements with that id but i do see ones with attributes formatted like this. have to get the [1] instance since 
      the first appears to be a thumbnail
      const pageDiv = document.querySelectorAll(`[data-page-number="${pageNumber}"]`)[1];
      if (!pageDiv) return;
      const canvas = pageDiv.querySelector('.canvasWrapper > canvas') as HTMLCanvasElement | null;
      if (!canvas) return;
      this.setupEventsForCanvas(canvas); // this is getting called with what seem to be the appropriate canvases
    }
  }

however this still is not allowing any of my listeners to hit when i click and drag on the screen... as a sanity check that im getting the correct canvas ive done

canvas.setAttribute('fake-attribute', 'canvas-${pageNumber}')

and i do see this showing up in the dom. its almost like my click/mousedown/mousemove/mouseup events are not being propogated to the canvas and are instead hitting some element wrapping them?

[stephanrauh]

Oh, yes, you're probably right:

• As for PDFViewerApplication: I thought you'd need it, but then I say the DOM-based approach and forgot to tidy up.
• As for blocking the mouse events: on top of the canvas are an optional text layer and an optional annotation layer. I guess your mouse events a swallowed by the text layer.
• "Not 100% understanding this as i have never actually used canvases before": sometimes it's difficult to explain simple things. I suppose you'll see the problem immediately once you've got your event listeners up an running.

[skyleguy]

I suppose I can try adding the listeners to the text layer element. I did try to turn text layer and annotation layer off (as users of my app should not need them as far as I know) before but I remember that they didn’t actually seem to be removed from the dom? Anyway, I’ll hopefully update this discussion with any solution I end up getting to work

[skyleguy]

Thanks for the advice! I was able to get this working out after a little while longer (did have to use AI to figure out that I was suffering from the concept of canvas scaling issues due to my retina screen where i was seeing my drag and drop not matching and the image i cut was scaled as well). But here is essentially what i have with a demo video

working-example.mov

onPageRendered(event: PageRenderedEvent) {
    const pageNumber = event.pageNumber;
    if (event.pageNumber) {
      if (!pageNumber) return;
      const pageDiv = document.querySelectorAll(`[data-page-number="${pageNumber}"]`)[1] as HTMLElement | null;
      if (!pageDiv) return;
      const canvas = pageDiv.querySelector('.canvasWrapper > canvas') as HTMLCanvasElement | null;
      if (!canvas) return;
      this.setupEventsForCanvas(canvas, pageDiv);
    }
  }

  private setupEventsForCanvas(canvas: HTMLCanvasElement, target: HTMLElement) {
    let startY = 0;
    let width = 0;
    let height = 0;
    let isDragging = false;
    const ctx = canvas.getContext('2d');
    if (ctx) {
      const originalCanvasImage = ctx.getImageData(0, 0, canvas.width, canvas.height);
      ctx.fillStyle = 'rgba(0, 0, 0, 0.3)';
      target.addEventListener('mousedown', (e) => {
        isDragging = true;
        startY = this.calculateYCoordinateAfterDisplayScale(canvas, e.clientY);
        console.log('starting at: ' + startY + ' of', canvas.height);
      });

      target.addEventListener('mousemove', (e) => {
        if (!isDragging) return;
        ctx.putImageData(originalCanvasImage, 0, 0);
        width = canvas.width;
        const currentY = this.calculateYCoordinateAfterDisplayScale(canvas, e.clientY);
        height = currentY - startY;
        ctx.fillRect(0, startY, width, height);
      });

      target.addEventListener('mouseup', () => {
        isDragging = false;
        if (height !== 0) {
          ctx.putImageData(originalCanvasImage, 0, 0);
          const imageData = ctx.getImageData(0, startY, width, height);
          if (imageData) {
            this.imageService.imageUrl.set(this.imageDataToDataUrl(imageData, width, height));
          }
        }
        startY = 0;
        width = 0;
        height = 0;
      });
    }
  }

  private calculateYCoordinateAfterDisplayScale(canvas: HTMLCanvasElement, clientY: number) {
    const rect = canvas.getBoundingClientRect();
    return (clientY - rect.top) * this.getCanvasScale(canvas);
  }

  private getCanvasScale(canvas: HTMLCanvasElement) {
    const rect = canvas.getBoundingClientRect();
    return canvas.height / rect.height;
  }

  private imageDataToDataUrl(imageData: ImageData, width: number, height: number): string {
    const scale = 1 / (window.devicePixelRatio || 1);
    const outputCanvas = document.createElement('canvas');
    outputCanvas.width = width * scale;
    outputCanvas.height = height * scale;

    const outputCtx = outputCanvas.getContext('2d');
    if (outputCtx) {
      const tempCanvas = document.createElement('canvas');
      tempCanvas.width = width;
      tempCanvas.height = height;
      const tempCtx = tempCanvas.getContext('2d');
      tempCtx?.putImageData(imageData, 0, 0);

      outputCtx.drawImage(tempCanvas, 0, 0, outputCanvas.width, outputCanvas.height);

      return outputCanvas.toDataURL();
    }
    return '';
  }