Awesome works and resources relevant to 360 processing. Insipred by awesome-360-vision.
Welcome to pull.
- Awesome-360-processing
-
SaltiNet: Scan-path prediction on 360 degree images using saliency volumes
Marc Assens, Kevin McGuinness, Xavier Giro-i-Nieto, Noel E. O'Connor
IEEE International Conference on Computer Vision Workshops (ICCVW), 2017.
[pdf] [code] -
The prediction of head and eye movement for 360 degree images
Zhu, Yucheng, Guangtao Zhai, and Xiongkuo Min
Signal Processing: Image Communication (SPIC), 2018.
[pdf] -
PathGAN: Visual scanpath prediction with generative adversarial networks
Marc Assens, Xavier Giro-i-Nieto, Kevin McGuinness, and Noel E. O’Connor
European Conference on Computer Vision (ECCV), 2018.
[pdf] [code] -
ScanGAN360: A generative model of realistic scanpaths for 360° images
Daniel Martin, Ana Serrano, Alexander W. Bergman, Gordon Wetzstein, Belen Masia
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2022.
[project] -
ScanDMM: A deep Markov model of scanpath prediction for 360° Images
Xiangjie Sui, Yuming Fang, Hanwei Zhu, Shiqi Wang, Zhou Wang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[pdf][project]
-
Saliency in VR: How do people explore virtual environments?
Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, Gordon Wetzstein
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2018.
[pdf][code] -
SalGAN360: Visual saliency prediction on 360 degree images with generative adversarial networks
Fang-Yi Chao, Lu Zhang, Wassim Hamidouche, and Olivier Deforges
IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2018.
[pdf] [code] -
SalNet360: Saliency maps for omnidirectional images with CNN
Rafael Monroy, Sebastian Lutz, Tejo Chalasani, and Aljosa Smolic
Signal Processing: Image Communication (SPIC), 2018.
[pdf] [code] -
A novel superpixel-based saliency detection model for 360-degree images
Fang, Yuming, Xiaoqiang Zhang, and Nevrez Imamoglu
Signal Processing: Image Communication (SPIC), 2018.
[pdf] [code] -
SalGCN: Saliency prediction for 360-degree images based on spherical graph convolutional networks
Haoran Lv, Qin Yang, Chenglin Li, Wenrui Dai, Junni Zou, Hongkai Xiong
ACM International Conference on Multimedia (ACMM), 2020.
[pdf] -
Stage-wise salient object detection in 360° omnidirectional image via object-level semantical saliency ranking
Guangxiao Ma, Shuai Li, Chenglizhao Chen, Aimin Hao, Hong Qin
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2020.
[pdf][code + dataset] -
A multi-FoV viewport-based visual saliency model using adaptive weighting losses for 360◦ images
Fang-Yi Chao, Lu Zhang, Wassim Hamidouche, Olivier Déforges
IEEE Transactions on Multimedia (TMM), 2020.
[pdf] [code] -
SalBiNet360: Saliency prediction on 360° images with local-global bifurcated deep network
Dongwen Chen, Chunmei Qing, Xiangmin Xu, Huansheng Zhu
IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2020.
[pdf]
-
A dataset of head and eye movements for 360 degree images [salient360!]
Yashas Rai, Jesús Gutiérrez, and Patrick Le Callet
ACM Multimedia Systems Conference (ACMMSys), 2017.
[pdf] [project] -
A fixation-based 360° benchmark dataset for salient object detection
Yi Zhang, Lu Zhang, Wassim Hamidouche, Olivier Deforges
IEEE International Conference on Image Processing (ICIP), 2020.
[pdf][dataset]
-
Pano2vid: Automatic cinematography for watching 360° videos
Yu-Chuan Su, Dinesh Jayaraman, and Kristen Grauman
Asian Conference on Computer Vision (ACCV), 2016.
[pdf] [code] [dataset] -
Making 360° video watchable in 2D: learning videography for click free viewing
Yu-Chuan Su, Kristen Grauman
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[pdf] [code] -
Deep 360 pilot: Learning a deep agent for piloting through 360° sports video
Hou-Ning Hu, Yen-Chen Lin, Ming-Yu Liu, Hsien-Tzu Cheng, Yung-Ju Chang, Min Sun
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[pdf] [code + dataset] -
Tell me where to look: Investigating ways for assisting focus in 360° video
Yen-Chen Lin, Yung-Ju Chang, Hou-Ning Hu, Hsien-Tzu Cheng, Chi-Wen Huang, Min Sun
Conference on Human Factors in Computing Systems (CHI), 2017.
[pdf] [project] -
Your attention is unique: Detecting 360-degree video saliency in head-mounted display for head movement prediction
Nguyen, Anh, Zhisheng Yan, and Klara Nahrstedt
ACM international conference on Multimedia (ACMM), 2018.
[pdf] [code] -
Predicting head movement in panoramic video: A deep reinforcement learning approach
Mai Xu, Yuhang Song, Jianyi Wang, Minglang Qiao, Liangyu Huo, and Zulin Wang
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019.
[pdf] [code] -
Interactive and automatic navigation for 360° video playback
Kyoungkook Kang and Sunghyun Cho
ACM Transactions on Graphics (TOG), 2019.
[pdf] [project] -
Attention-based deep reinforcement learning for virtual cinematography of 360° videos
Jianyi Wang, Mai Xu, Lai Jiang, Yuhang Song
IEEE Transactions on Multimedia (TMM), 2020.
[pdf] -
DGaze: CNN-based gaze prediction in dynamic scenes
Zhiming Hu, Sheng Li, Congyi Zhang, Kangrui Yi, Guoping Wang, Dinesh Manocha
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2020.
[pdf] [code] [project] -
A spherical convolution approach for learning long term viewport prediction in 360 immersive video
Chenglei Wu, Rui-Xiao Zhang, Zhi Wang, Lifeng Sun
AAAI Conference on Artificial Intelligence (AAAI), 2020.
[pdf] [code] -
Transitioning360: Content-aware NFoV virtual camera paths for 360° video playback
Miao Wang, Yi-Jun Li, Wen-Xuan Zhang
International Symposium on Mixed and Augmented Reality (ISMAR), 2020.
[pdf] -
Scanpath prediction in panoramic videos via expected code length minimization
Mu Li, Kanglong Fan, and Kede Ma
arXiv:2305.02536, 2023.
[pdf]
-
Cube padding for weakly-supervised saliency prediction in 360° videos
Hsien-Tzu Cheng, Chun-Hung Chao, Jin-Dong Dong, Hao-Kai Wen, Tyng-Luh Liu, Min Sun
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[pdf] [code] -
V-BMS360: A video extention to the BMS360 image saliency model
Lebreton, Pierre, Stephan Fremerey, and Alexander Raake
IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2018.
[pdf] [code] -
Saliency detection in 360° videos
Ziheng Zhang, Yanyu Xu, Jingyi Yu
European Conference on Computer Vision (ECCV), 2018.
[pdf] [code] -
Video Saliency Prediction Based on Spatial Temporal Two-Stream Network
Kao Zhang and Zhenzhong Chen
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2019.
[pdf] [code] -
Viewport-dependent saliency prediction in 360° video
Minglang Qiao, Mai Xu, Zulin Wang, Ali Borji
IEEE Transactions on Multimedia (TMM), 2020.
[pdf] -
ATSal: An attention based architecture for saliency prediction in 360° videos
Yasser Dahou, Marouane Tliba, Kevin McGuinness, Noel O'Connor
arxiv, 2020.
[pdf] [code]
- A saliency dataset for 360-degree videos
Anh Nguyen and Zhisheng Yan
ACM Multimedia Systems Conference (ACMMSys), 2019.
[pdf] [project]
-
[WS-PSNR] Weighted-to-spherically-uniform quality evaluation for omnidirectional video
Yule Sun, Ang Lu, Lu Yu
IEEE Signal Processing Letters (SPL), 2017.
[pdf] -
[W-SSIM/WMS-SSIM] Subjective and objective quality assessment of omnidirectional video
Francisco Lopes, João Ascenso, António Rodrigues, Maria Paula Queluz
Applications of Digital Image Processing, 2018.
[pdf] -
[CPP-PSNR] Quality metric for spherical panoramic video
Vladyslav Zakharchenko, Kwang Pyo Choi, Jeong Hoon Park
Optics and Photonics for Information Processing, 2016.
[pdf] -
Deep virtual reality image quality assessment with human perception guider for omnidirectional image
Hak Gu Kim, Heoun-Taek Lim, Yong Man Ro
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2019.
[pdf]
-
[S-PSNR] A framework to evaluate omnidirectional video coding schemes
Matt Yu, Haricharan Lakshman, Bernd Girod
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2015.
[pdf] -
[S-SSIM] Spherical structural similarity index for objective omnidirectional video quality assessment
Sijia Chen, Yingxue Zhang, Yiming Li, Zhenzhong Chen, Zhou Wang
IEEE International Conference on Multimedia and Expo (ICME), 2018.
[pdf]
-
Saliency-driven omnidirectional imaging adaptive coding: Modeling and assessment
Guilherme Luz, João Ascenso, Catarina Brites, Fernando Pereira
International Workshop on Multimedia Signal Processing (MMSP), 2017.
[pdf] -
Assessing visual quality of omnidirectional videos
Mai Xu, Chen Li, Zhenzhong Chen, Zulin Wang, Zhenyu Guan
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2019.
[pdf] -
Quality assessment of stereoscopic 360-degree images from multi-viewports
Jiahua Xu, Ziyuan Luo, Wei Zhou, Wenyuan Zhang, Zhibo Chen
IEEE Picture Coding Symposium (PCS), 2019.
[pdf] -
Bridge the gap between VQA and human behavior on omnidirectional video: A large-scale dataset and a deep learning model
Chen Li, Mai Xu, Xinzhe Du, Zulin Wang
ACM international conference on Multimedia (ACMM), 2018.
[pdf] [dataset] -
Viewport proposal CNN for 360° video quality assessment
Chen Li, Mai Xu, Lai Jiang, Shanyi Zhang, Xiaoming Tao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[pdf] [code] -
MC360IQA: A multi-channel CNN for blind 360-degree image quality assessment
Wei Sun, Weike Luo, Xiongkuo Min, Guangtao Zhai, Xiaokang Yang, Ke Gu, Siwei Ma
IEEE Journal of Selected Topics in Signal Processing (JSTSP), 2019.
[pdf] -
Blind omnidirectional image quality assessment with viewport oriented graph convolutional networks
Jiahua Xu, Wei Zhou, Zhibo Chen
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2020.
[pdf] -
Perceptual quality assessment of omnidirectional images as moving camera videos
Xiangjie Sui, Kede Ma, Yiru Yao, Yuming Fang
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021.
[pdf] [code] -
Viewport-based omnidirectional video quality assessment: Database, modeling and inference
Yu Meng, Zhan Ma
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2021.
[pdf] [project] -
Omnidirectional image quality assessment by distortion discrimination assisted multi-stream network
Yu Zhou, Yanjing Sun, Leida Li, Ke Gu, and Yuming Fang
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2022. [pdf] -
Perceptual quality assessment of omnidirectional images
Yuming Fang, Liping Huang, Jiebin Yan1, Xuelin Liu, Yang Liu
AAAI Conference on Artificial Intelligence (AAAI), 2022.
[pdf] -
Subjective quality assessment of user-generated videos
Yuming Fang, Yiru Yao, Xiangjie Sui, and Kede Ma
IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2023.
[pdf][Database] -
Perceptual quality assessment of virtual reality videos in the wild
Wen Wen, Mu Li, Yiru Yao, Xiangjie Sui, Yabin Zhang, Long Lan, Yuming Fang, Kede Ma
arXiv:2206.08751, 2022.
[pdf] -
Assessor360: Multi-sequence network for blind omnidirectional image quality assessment
Tianhe Wu, Shuwei Shi, Haoming Cai, Mingdeng Cao, Jing Xiao, Yinqiang Zheng and Yujiu Yang
arXiv:2305.10983, 2023.
[pdf][code] -
Perceptual quality assessment of 360° images based on generative scanpath representation
Xiangjie Sui, Hanwei Zhu, Xuelin Liu, Yuming Fang, Shiqi Wang, Zhou Wang
arXiv:2309.03472, 2023.
[pdf][code]
-
VRSA net: VR sickness assessment considering exceptional motion for 360° VR video
Hak Gu Kim, Heoun-Taek Lim, Sangmin Lee, Yong Man Ro
IEEE Transactions on Image Processing (TIP), 2018.
[pdf] -
VR sickness versus VR presence: A statistical prediction model
Woojae Kim, Sanghoon Lee, Alan Conrad Bovik
IEEE Transactions on Image Processing (TIP), 2020.
[pdf] -
Assessing individual VR sickness through deep feature fusion of VR video and physiological response
Sangmin Lee, Seongyeop Kim, Hak Gu Kim, Yong Man Ro
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2021.
[pdf]
-
Graph learning based head movement prediction for interactive 360 video streaming
Xue Zhang, Gene Cheung, Yao Zhao, Patrick Le Callet, Chunyu Lin, and Jack Z. G. Tan
IEEE Transactions on Image Processing (TIP), 2021.
[pdf] -
QoE evaluation methods for 360-degree VR video transmission
Zesong Fei, Fei Wang, Jing Wang, Xiang Xie
IEEE Journal of Selected Topics in Signal Processing (JSTSP), 2020.
[pdf] -
A Log-Rectilinear transformation for foveated 360-degree video streaming
David Li, Ruofei Du, Adharsh Babu, Camelia Brumar, and Amitabh Varshney
IEEE Transactions on Visualization and Computer Graphics (TVCG Honorable Mentions), 2021.
[pdf] [project] [code]