Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to improve performance when running tfjs in web worker? #7896

Open
libofei2004 opened this issue Aug 9, 2023 · 0 comments
Open

how to improve performance when running tfjs in web worker? #7896

libofei2004 opened this issue Aug 9, 2023 · 0 comments

Comments

@libofei2004
Copy link

libofei2004 commented Aug 9, 2023

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MIUI13.0.5 android12
TensorFlow.js installed from (npm or script link): Script Links
TensorFlow.js version (use command below): 4.9.0
Browser version: webview 96

I run tfjs to do object detecting in webview on android mobile phone. For improving the performance , I usually use 6 workers to do object detecting job, but on usual the more workers are executed, the worse performance for one worker I met.

If I use only 1 worker to detect, the waste time is about 60ms - 100ms. If I use 6 worker to detect, the waste time in 1 worker is about 200-300ms in most cases, occasionally the waste time in 1 worker may be about 60ms - 100ms. I don't know how to improve to obtain a stable high performance like 60ms - 100ms when I use 6 workers?
my main code:
index.js

	     ...
             var canvasElement = new OffscreenCanvas(clientWidth, clientHeight);
             function detectImage() {
                         ...                       
			let ctx2 = canvasElement.getContext("2d");
			ctx2.drawImage(V, 0, 0, clientWidth, clientHeight);
                       var img = canvasElement.transferToImageBitmap();
			pool.push(img);
                        setTimeout(function () {
					detectImage();
			}, 40);
              }

              function doDetect(){
                        ...
                        let workerShell = getWorkerPool(); 			
			if(workerShell){
                            let img = pool.shift();
			   doWork(workerShell, img, dealResult);
                        }
              }
            
              function initWorkerPool(num){
		     workerPool = [];
		     for(let i=0;i<num;i++){
			     let worker = new Worker("work.js");
			     workerPool.push({
				    num:i+1,
				    worker:worker,
					busy:false
				 });	
				 worker.postMessage({
						action: "model"
				 });				 
		   }
		}

               function doWork(workerShell, imgdata, callback){
		        let worker = workerShell.worker;
			workerShell.busy = true;		
			worker.onmessage = function(e) {
				let result = e.data.result;			 
				callback(result);
				workerShell.busy = false;
			};		
         	
			worker.postMessage({
					image: imgdata,
					action: "image"
			}, [imgdata]);
		}
		
               function animate() {
			requestAnimationFrame( animate );
			doDetect();
		}
               //simplify code , actually I will wait for all models are loaded in workers and click a button to run detectImage()
                ...
		initWorkerPool(6);
                detectImage();
		animate();

worker.js

importScripts("tf.min4.9.js");
//customvision-tfjs
importScripts("index.umd.js");

var model = new cvstfjs.ObjectDetectionModel();

async function loadModel(){
    await model.loadModelAsync('TensorFlowJs_g_It21/model.json');   
}

self.addEventListener('message', async function (e) {
    
  var action = e.data.action;
  if(action=="image"){
	 var imagedata = e.data.image;
	 var start =  Date.now();
         var result = await model.executeAsync(imagedata);
         var end =  Date.now();
        var time = end-start;
// 60ms - 100ms in only 1 worker, 200-300ms in 6 workers.
        console.log("time",time);
        self.postMessage({result:result, time:time});
  }else if(action=="model"){
	 await loadModel();
	 var result = {state:1};
        self.postMessage(result);
  }
}

I use https://github.xiaoc.cn/microsoft/customvision-tfjs to loadmodel and detect object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants