Research fields

 

Autonomous Driving: Perception-Driving in Real Traffics 2009~present
 
Our goal is to develop computational cognitive models and systems, which can perceptually process sensing information of dynamic environment via multiple  heterogeneous sensors , and improve the efficiency and perceptual quality of the perception system for a unmanned vehicle. We focus on incorporating into the system with mechanisms including selective attention, visual feedback, and structural information of spatial topology, etc. Furthermore, we develop intelligent motion decision-making methods as well as efficient perception-action models  for autonomous driving. Our research covers the following topics:
1) Computational framework for the perception of environment for unmanned vehicle, which has functions of selective attention and interactive fusion of multiple sensors;
2) Dynamical and collaborative computing model for sensing local dynamic scenarios;
3) Knowledge representation and reasoning of driving behaviors;
4) Real-time and highly reliable motion planning and controller for unmanned vehicle.
 
 

 

 

 

Selected Publications

 

  1. PuZhang, LeiBai, YuningWang, JianwuFang, Jianru Xue, NanningZheng, WanliOuyang, Towards Trajectory Forecasting From Detection. IEEE Trans. Pattern Anal. Mach. Intell.45(10): 12550-12561 (2023).

  2. Pu Zhang, Jianru Xue, Pengfei Zhang, Nanning Zheng, Wanli Ouyang, Social-Aware Pedestrian Trajectory Prediction via States Refinement LSTM. IEEE Trans. Pattern Anal. Mach. Intell. 44(5): 2742-2759 (2022).

  3. Xingyu Chen, Jianru Xue, Shanmin Pang, Sparse Semantic Map-Based Monocular Localization in Traffic Scenes Using Learned 2D-3D Point-Line Correspondences. IEEE Robotics Autom. Lett. 7(4): 11894-11901 (2022).

  4. , , , Jianru Xue, FEND: A Future Enhanced Distribution-Aware Contrastive Learning Framework for Long-Tail Trajectory Prediction. CVPR 2023: 1400-1409

  5. , Jianru Xue, , CalibDepth: Unifying Depth Map Representation for Iterative LiDAR-Camera Online Calibration. ICRA 2023: 726-733

  6. Yuxin Pan, Jianru Xue, Wanli Ouyang, Pengfei Zhang, Jianwu Fang, Xingyu Chen, and Pu Zhang, Navigation Command Matching for Vision-based Autonomous Driving, ICRA 2020.

  7. Pengfei Zhang,Jianru Xue, Guilin Lan, Wenjun Zeng, Jianru Xue, Nanning Zheng, Zhanning Gao, EleAtt-RNN: Adding Attentiveness to Neurons in Recurrent Neural Networks, IEEE Transactions on Image Processing, 2020,29(1):1061-1073.

  8. Jianru Xue, Jianwu Fang, Tao Li, Bohua Zhang, Pu Zhang, Zhen Ye, Jian Dou,  BLVD: Build A Large-scale 5D Semantics Benchmark for Autonomous Driving, IEEE International Conference on Robotics and Automation, ICRA 2019.

  9. Jian Dou, Jianru Xue, Jianwu Fang,  SEG-VoxelNet for 3D Vehicel Detection from RGB and LiDAR Data, IEEE International Conference on Robotics and Automation, ICRA 2019.

  10. Di Wang, Jianru Xue, Wei Zhan, Yinghan Jin, Nanning Zheng, Masayoshi Tomizuka, Prcise Cottentropy-based 3D Object Modelling with Geometric Traffic Prior, IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019.

  11. Pu Zhang, Wanli Ouyang, Pengfei Zhang, Jianru Xue, Nanning Zheng, SR: LSTM State Refinement for Pedestrian Trajectory Predicition, IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019.

  12. Di Wang, Jianru Xue, Zhongxin Tao, Yang Zhong, Dixiao Cui, Shaoyi Du, Accurate Mix-Norm-based Scan Matching via Residual Error Modelling, IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018.

  13. Chao Ma, Jianru Xue, Yuehu Liu, Jing Yang, Yongqiang Li, Nanning Zheng, Data-driven state-increment statistical model and its application in autonomous driving, IEEE Transactions on Intelligent Transportation Systems, 2018, 19(12): 3872-3882.

  14. Jianru Xue, Jianwu Fang, Pu Zhang, A Survey of Scene Understanding by Event Reasoning in Autonomous Driving, International Journal of Automation and Computing, 2018, 15(3):249-266. (Invited paper).

  15. Jianru Xue, Di Wang, Shaoyi Du, Dixiao Cui, Yong Huang, Nanning Zheng., A vision-centered multi-sensor fusing approach to self-localization and obstacle perception for robotic cars. Frontiers of IT & EE , 2017, 18(1): 122-138. (Invited paper).

  16. Dixiao Cui, Jianru Xue, Nanning Zheng, Real-Time Global Localization of Robotic Cars in Lane Level via Lane Marking Detection and Shape Registration. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(4): 1039-1050.

  17. Liang Ma, Jianru Xue, Kuniaki Kawabata, Jihua Zhu, Chao Ma, Nanning Zheng, Efficient Sampling-based Motion Planning for On-road Autonomous Driving, IEEE Transactions on Intelligent Transportation Systems. 2015, 16(4):1961-1976.

  18. Kuniaki Kawabata,Liang Ma,Jianru Xue, Sho Yokota,Yasue Mitsukura, Nanning Zheng, A Path Generation for Automated Vehicle based on Bezier Curve and Via-points, Robotics and Autonomous Systems, 2015, 74:243-252.

  19. Dixiao Cui, Jianru Xue, Shaoyi Du, Nanning Zheng, Real-time Global Localization of Intelligent Road Vehicels in Lane-level via Lane Marking Detection and Shape Registration,  IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2014.

Visual Pattern Analysis for Scene Understanding, Prediction, and Coding. 2002~present

 

We focus on the problem of perceptual grouping of visual data. We try to develop computational models to simulate mechanisms of human visual systems, and develop the perceptual mechanism-based models and algorithms. We aim to develop a series of models and methods for pattern recognition, scene understanding, prediction and coding. We hope that these models and methods can reduce the performance gap between human visual system and machine. More specifically, our research covers topics as follows:
1) Video parsing model based on visual cognitive mechanism;
2) Parsimonious representation model;
3) Joint Source and Channel coding for robust and security video applications.
 

 Selected Publications

  1. PengfeiZhang, CuilingLan, WenjunZeng, JunliangXing, JianruXue, NanningZheng, Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition, CVPR 2020.

  2. Pengfei Zhang,Guilin Lan, Junliang Xing, Wenjun Zeng, Jianru Xue, Nanning Zheng, View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(8):1963-1978. 

  3. Le Wang, Gang Hua, Rahul Sukthankar, Jianru Xue, Zhenxing Niu, Nanning Zheng, Video Object and Co-Segmentation with Extremely Weak Supervision, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(10): 2074-2087.

  4. Shanmin Pang, JIanru Xue, Jihua Zhu, Li Zhu, Qi Tian, Unfiying Sum and Weighted Aggregations for Efficient Yet Effective Image Representation Computation, IEEE Transactions on Image Processing, 2019, 28(2): 841-852.

  5. Shanmin Pang, Jin Ma, Jianru Xue, Jihua Zhu, Vicente Ordonez, Deep feature aggregation and image re-ranking with heat diffusion for Image Retrieval, IEEE Transactions on Multimedia, 2019, 21(6):1513-1523.

  6. Shanmin Pang, Jin Ma, Jihua Zhu, Jianru Xue, Qi Tian, Improving Object Retrieval Quality by Integration of Similarity Propagation and Query Expansion, IEEE Transactions on Multimedia, 2019, 21(3):760-770.

  7. Shanmin Pang, Jihua Zhu, Jiaxing Wang, Vicente Ordonez, Jianru Xue, Building Discriminative CNN Image Representations for Object Retrieval Using the Replicator Equation, Pattern Recognition, 2018, 83:150-160. 

  8. Zhanning Gao, Jianru Xue, Wengang Zhou, Qi Tian, Democratic Diffusion Aggregation for Image Retrieval, IEEE Transactions on Multimedia, 2016, 18(8):1661-1674.

  9. Xiaofeng Wang, Kemu Pang, Xiaorui Zhou, Yang Zhou, Lu Li, Jianru Xue, A Visual Model based Perceptual Image Hash for Content Authentication, IEEE Transactions on Information Forensics and Security, 2015, 10(7):1336-1349.

  10. Le Wang, Gang Hua, Jianru Xue, Zhanning Gao, Nanning Zheng, Joint Segmentation and Recognition of Categorized Objects from Noisy Web Image Collection, IEEE Transactions on Image Processing, 2014, 23(9):4070-4086.

  11. Jianru Xue, Le Wang, Nanning Zheng, and Gang Hua, Automatic Salient Object Extraction with Contextual Cue and Its Applications to Recognition and Alpha Matting, Pattern Recognition, 2013, 46(11):2874-2289.

  12. Hu, W., Xue, J.,Zheng, N., PSF Estimation via Gradient Domain Correlation, IEEE Trans. on Image Processing, 2012, 21(1): 386-392.

  13. Jianru Xue, Ce Li, and Nanning Zheng., Proto-object based Rate Control for JPEG2000: An Approach to Content-based Scalability, IEEE Transactions on Image Processing, 2011, 20(4): 1177-1184.

  14. 薛建儒，郑南宁，权炜，自适应分割的雷达回波数据混合编码算法，《电子与信息学报》，2003, 25(8):1059-1065.

  15. 薛建儒，郑南宁，张元林, 雷达回波无损编码，《西安交通大学学报》,2002, 3610, pp.1070-1074.
Visual Tracking Using Statistical Approaches 2002~present
 
Motion perception is one of essential functions of mammal vision, and building motion perception model that has a reasonable biologic basis plays a very important role in computer vision. Based on the Bayesian machine (Bayesian analysis + MCMC computation) and statistical learning, We focus on constructing nonlinear representation models for motion information in video with probabilistic graphical model. As to the task of model learning and inference, we imports the idea of 'survival of the fittest' in simulated evolution computing to design an efficient statistical evolution computing method. Furthermore, based on the motion representation model and computing method, we focus on three problems in visual tracking:
1)    Multi-target tracking;
2)    Nonlinear dynamics classifying and learning, Motion analysis and synthesis.

 

 Selected Publications

  1. Jianru Xue, Nanning Zheng, Jason Geng, Xiaopin Zhong, Tracking Multiple Visual Targets via Particle-based Belief Propagation, IEEE Transactions on System, Man, and Cybernetics Part B, 2008, 38(1) :196-209.
  2. Jianru Xue, Zheng Ma, Nanning Zheng, Hierarchical Model for Joint Detection and Tracking of Multi-target, 10th Asian Conference on Computer Vision (ACCV2009), 2009.
  3. Jianru Xue, Nanning Zheng, Xiaopin Zhong, Visual Perceptual Stimulus—A Bayesian-based Integration of Multi-Visual-Cue Approach and Its Application, China Science Bulletin, 2008, 53(2):172-182.
  4. Xue Jianru, Zheng Nanning, Zhong Xiaopin., Sequential Stratified Sampling Belief Propagation for Multiple Targets Tracking, Science in China, Issue. F, 2006, 49(1): 48-62.
  5. Jianru Xue, Nanning Zheng. Robust Tracking with and Beyond Visible Spectrum: A Four-Layer Data Fusion Framework. International Workshop on Intelligence Computing in Pattern Analysis/Synthesis (IWICPAS2006), Springer LNCS 4153 proceeding, pp. 1-16, 2006.
  6.  Jianru Xue, Nanning Zheng, An integrated Monte Carlo data association framework for multi-object tracking, International conference on Pattern Recognition, 2006.

  7. Jianru Xue，Nanning Zheng, Xiaopin Zhong, Tracking Targets via Particle based Belief Propagation, 7th Asian Conference on Computer Vision 2006, Springer LNCS3851, Part I: 348-358, 2006.

  8. Xiaopin Zhong, Jianru Xue and Nanning Zheng. Graphical Model based Cue Integration Strategy for Head Tracking. In Proceedings of 17th British Machine Vision Conference, 2006.

  9. Xiaopin Zhong, Nanning Zheng, and Jianru Xue, Pseudo Measurement based Multiple Model Approach for Robust Player Tracking, 7th Asian Conference on Computer Vision 2006, Springer LNCS 3852, Part II:781-790, 2006.

  10. 钟小品，薛建儒，郑南宁，平林江，基于融合策略自适应的多线索跟踪方法，《电子与信息学报》，2007, 29(5):1017-1022.

  11. 薛建儒，郑南宁，多目标跟踪的序贯分层抽样信任传播算法，《中国科学》.E辑，2005, 35(10): 1049－1063.

  12. 薛建儒，郑南宁，郑朝晖，权炜，基于自适应高斯混合体模型的相控阵雷达TWS跟踪技术，《电子学报》，2003, 31(3):433-436.