* Currently, English is used if we do not have translations

CyborgCrowd: Flexible and Scalable Aggregation of Human-Machine Intelligence

This page is a tentative version and to be updated.

What's new

May 3rd, 2017: We will have the JST CREST Three Project Joint Symposium "Better Future of People and AI" at University of Tsukuba on the 2nd of June

1. Overview

In this project, we conduct research on theory and implementation of middleware that achieves flexible integration of crowdsourced processing, machine processing and any combination of them, taking the availability of people and algorithms into consideration. We also conduct feasibility studies in various application domains including disaster response, aiming to establish flexible and scalable infrastructure that aggregates human and machine intelligence. This project is conducted as a part of the "Intelligent Information Processing Systems Creating Co-Experience Knowledge and Wisdom with Human-Machine Harmonious Collaboration" research area under the supervision of Dr. Norihiro Hagita (Board Director, Director, Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute International) The grant period starts on the 1st of Dec., 2016 and ends on the 31st of March, 2022.

2. People

Research Group Leaders

  • Atsuyuki Morishima (Principal Investigator, Middleware Resaearch Group Co-Leader, University of Tsukuba)
  • Itaru Kitahara (Middleware Research Group Co-Leader, University of Tsukuba)
  • Munenari Inoguchi (Society Implementation Research Group Leader, Shizuoka University)
  • Keishi Tajima (Theory Research Group Leader, Kyoto University)

Research Participants and Collaborators

  • Sihem Amer-Yahia (CNRS/LIG)
  • Eiji Aramaki (NAIST)
  • Masayoshi Aritsugi (Kumamoto University)
  • Yukino Baba (Kyoto University)
  • Senjuti Basu Roy (NJIT)
  • Takashi Harada (Doshisha University)
  • Rumi Hiraga (Tsukuba University of Technology)
  • Kosetsu Ikeda (Chiba University)
  • Kaori Ishii (University of Tsukuba)
  • Yutaka Ito (Akita University)
  • Hisashi Kashima (Kyoto University)
  • Youhei Kawamura (Akita University)
  • Masaaki Koido (University of Tsukuba)
  • Kiyonori Nagasaki (International Institute for Digital Humanities)
  • Akira Sato (University of Tsukuba)
  • Tetsuo Sakaguchi (University of Tsukuba)
  • Nobuyuki Shimizu (Yahoo!Japan Research)
  • Takeaki Shionome
  • Yuhki Shiraishi (Tsukuba University of Technology)
  • Hidehiko Shishido (University of Tsukuba)
  • Shigeo Sugimoto (University of Tsukuba)
  • Sho Tamaki (Meio University)
  • Shoko Wakamiya (NAIST)
  • Daisuke Wakatsuki (Tsukuba University of Technology)
  • Eiichi Yamazaki (Kansai University)
  • Jianwei Zhang (Iwate University)

Research Administrators

  • Kazuyo Tanaka (University of Tsukuba)
  • Ai Miyata (University of Tsukuba)

Technical Staff

  • Kinuyo Tarumoto


  • Yasue Shinozaki
  • Nobuko Hitomi
  • Hitomi Kimura
  • Kaori Sato

3. Collaborating Organizations

4. Research Groups

Middleware Research Group: University of Tsukuba

This group conducts research on (1) the CyborgCrowd computation model and (2) the requirements, design, and implementation of the CyborgCrowd middleware, collaborating with top leaders of real-world appllication domains. The model must be carefully designed taking into consideration the tradeoff between its expressive power and usability, while middleware must be extensible so that it can handle solutions required in the real-world problems.

Society Implementation Research Group: Shizuoka University

This group conducts research on CyborgCrowd from the viewpoint of real-world applications, focusing on the requirements to the CyborgCrowd middleware and its deployment into various applications. In particular, the group addresses applications that require geographical maps and aerial shoots in order to develop Common Operational Picture which is necessary for effective disaster response, because such applications require humans' cognitive ability and thus are a promising domain where crowdsourcing is effective. For achievement of applicating CyborgCrowd to such complicated real-world, we have to re-design work-flow of information management integrating human-resources and machines rationally. The group identifies the requirements for the middleware and benchmark scenarios for its evaluation, so that the project can design the CyborgCrowd middleware in an appropriate way and make the middleware meet the required level of real-world applications.

Theory Research Group: Kyoto University

This group conducts theoretical research on (1) the solution space including human-based, machine-based and hybrid solutions, and (2) optimization of solutions in the space. Examples for the first issue include theoretical analysis of the expressibility and completeness of the CyborgCrowd computation model. Examples for the second issue include extending database optimimization techniques so that they can deal with data processing conducted by crowds, aiming at improving the data quality and performance of execution plans with complex data flows involving people.

5. Projects with Application-Domain Top Leaders

To make our research results meaningful, we always assume real-world applications when conducing our research. Our project has research participants and collaborators who are top leaders of important domains and conduct research projects toghther with researchers on fundermental issues.

Note: This list is under construction and incomplete.

6. Publications

Journal & Conference Papers (reviewed)

  1. Kosetu Ikeda, Ryota Hayashi, Kiyonori Nagasaki, Atsuyuki Morishima: Human-assisted OCR of Japanese Books with Different kinds of Microtasks, iConference 2017, Wuhan, Kapan, 5pages, March 22-25, 2017.
  2. Ryota Hayashi, Nobuyuki Shimizu, Atsuyuki Morishima, "Worker viewpoint: Valuable feedback for microtask designers in crowdsourcing," First International Workshop on Social Computing(IWSC 2017) co-organized with the 26th World Wide Web Conference, pp.1387-1393, Apr. 2017.
  3. Yuhki Shiraishi, Jianwei Zhang, Daisuke Wakatsuki, Katsumi Kumai, Atsuyuki Morishima, "Crowdsourced real-time captioning of sign language by deaf and hard-of-hearing people", International Journal of Pervasive Computing and Communications, Vol. 13 Issue: 1, pp.2-25, 2017.
  4. Hidehiko Shishido, Yoshinari Kameda, Yuichi Ohta, Itaru Kitahara, "Visual Tracking Method of a Quick and Anomalously Moving Badminton Shuttlecock," ITE Transactions on Media Technology and Applications (MTA), Vol.5, No.3, pp.110-120, (2017/7).
  5. Hidehiko Shishido, Aoi Harazaki, Yoshinari Kameda, Itaru Kitahara, "Smoothly Switching Method of Asynchronous Multi-View Videos Using Frame Interpolation", 3DTV Conference 2017: Research and Applications in Future 3D (3DTV-CON),2017, 4 pages, (2017/6).
  6. "Katsumi Kumai, Jianwei Zhang, Yuhki Shiraishi, Daisuke Wakatsuki, Hiroyuki Kitagawa and Atsuyuki Morishima, “Group Rotation Management in Real-Time Crowdsourcing”, The 19th International Conference on Information Integration and Web-based Applications & Services (iiWAS2017), pp 23-31, Wyndham Grand Salzburg Conference Centre Hotel, Salzburg, Austria, December 4-6, 2017."
  7. "Yuzuki Furuhashi, Masaki Matsubara, and Atsuyuki Morishima, “Crowd-based Best-effort Number Estimation”, The First IEEE Workshop on Human-Machine Collaboration in BigData (HMData2017), pp 4293-4295, The Westin Copley Place, Boston, USA, December 11th, 2017."
  8. "Naoki Kobayashi, Masaki Matsubara, Keishi Tajima, and Atsuyuki Morishima, “A Crowd-in-the- Loop Approach for Generating Conference Programs with Microtasks”, The First IEEE Workshop on Human-Machine Collaboration in BigData (HMData2017), pp 4312-4314, The Westin Copley Place, Boston, USA, December 11th, 2017."
  9. Hidehiko Shishido, Kazuki Yamanaka, Yoshinari Kameda, Itaru Kitahara, "Pseudo-Dolly-In Video Generation Combining 3D Modeling and Image Reconstruction," ISMAR 2017 Workshop on 13th: Workshop on Highly Diverse Cameras and Displays for Mixed and Augmented Reality (HDCD4MAR2017), pp.327-333, (2017/10).
    Hidehiko Shishido, Yutaka Ito, Youhei Kawamura, Toshiya Matsui, Atsuyuki Morishima, Itaru Kitahara, "Proactive Preservation of World Heritage by Crowdsourcing and 3D Reconstruction Technology," The First IEEE Workshop on Human-Machine Collaboration in BigData (HMData2017), pp.4426-4428, (2017/12).
  10. Koyo Kobayashi, Hidehiko Shishido, Yoshinari Kameda, Itaru Kitahara, "A Method to Generate Disaster-Damage Map by Using 3D photometry and Crowd Sourcing," The First IEEE Workshop on Human-Machine Collaboration in BigData (HMData2017), pp.4397-4399, (2017/12).
    Hidehiko Shishido, Itaru Kitahara, "Calibration Method for Sparse Multi-view Cameras by Bridging with a Mobile Camera," International Conference on Image Processing Theory, Tools and Applications (IPTA2017), 6 pages, (2017/11).
  11. Hidehiko Shishido, Itaru Kitahara, "Calibration Method for Sparse Multi-view Cameras by Bridging with a Mobile Camera," International Conference on Image Processing Theory, Tools and Applications (IPTA2017), 6 pages, (2017/11).
  12. "Takeaki Shionome, Hirotaka Hashimoto, Jianwei Zhang, Yuhki Shiraishi, Daisuke Wakatsuki, Atsuyuki Morishima and Yohei Seki, ""Complement of Incomplete Task Results for Real-time Crowdsourcing Interpretation"", Proc. The 21th International Conference on Asian Language Processing (IALP 2017), pp. 359-362, Singapore, December 2017."
  13. Xiaoni Duan, Keishi Tajima, "Improving Classification Accuracy in Crowdsourcing through Hierarchical Reorganization", The First IEEE Workshop on Human-Machine Collaboration in BigData (HMData2017), pp.4290-4292 (2017/12).

Press Releases

(to appear)