{"id":325,"date":"2024-08-21T01:24:14","date_gmt":"2024-08-21T01:24:14","guid":{"rendered":"https:\/\/neuromorphicrobotics.com\/?p=325"},"modified":"2024-08-21T01:24:14","modified_gmt":"2024-08-21T01:24:14","slug":"how-to-build-a-brain-inspired-vision-chip-with-complementary-pathways-for-open-world-sensing","status":"publish","type":"post","link":"https:\/\/braininspiredrobotics.com\/?p=325","title":{"rendered":"How to build a brain-inspired vision chip with complementary pathways for open-world sensing?"},"content":{"rendered":"<p style=\"text-align: justify;\">Zheyu Yang, Taoyi Wang, Yihan Lin, Yuguo Chen, Hui Zeng, Jing Pei, Jiazheng Wang, Xue Liu, Yichun Zhou, Jianqiang Zhang, Xin Wang, Xinhao Lv, Rong Zhao &amp; Luping Shi. <a href=\"https:\/\/www.nature.com\/articles\/s41586-024-07358-4\"><strong>A vision chip with complementary pathways for open-world sensing<\/strong><\/a>. Nature 629, 1027\u20131033 (2024). https:\/\/doi.org\/10.1038\/s41586-024-07358-4<\/p>\n<p style=\"text-align: justify;\">Abstract<br \/>\n&#8220;<strong><span style=\"color: #ff0000;\">Image sensors face substantial challenges when dealing with dynamic, diverse and unpredictable scenes in open-world applications<\/span><\/strong>. However, the development of image sensors towards <strong><span style=\"color: #ff0000;\">high speed, high resolution, large dynamic range and high precision is limited by power and bandwidth<\/span><\/strong>. Here we present <strong><span style=\"color: #ff0000;\">a complementary sensing paradigm inspired by the human visual system<\/span> <\/strong>that involves parsing visual information into <strong><span style=\"color: #ff0000;\">primitive-based representations and assembling these primitives to form two complementary vision pathways<\/span><\/strong>: a <strong><span style=\"color: #ff0000;\">cognition-oriented pathway<\/span><\/strong> for accurate cognition and an <strong><span style=\"color: #ff0000;\">action-oriented pathway for rapid response<\/span><\/strong>. To realize this paradigm, <strong><span style=\"color: #ff0000;\">a vision chip called Tianmouc is developed<\/span><\/strong>, incorporating a hybrid pixel array and a parallel-and-heterogeneous readout architecture. Leveraging the characteristics of the complementary vision pathway, Tianmouc achieves high-speed sensing of up to <strong><span style=\"color: #ff0000;\">10,000\u2009fps, a dynamic range of 130\u2009dB<\/span> <\/strong>and an advanced figure of merit in terms of spatial resolution, speed and dynamic range. Furthermore, <strong><span style=\"color: #ff0000;\">it adaptively reduces bandwidth by 90%<\/span><\/strong>. We demonstrate the<strong><span style=\"color: #ff0000;\"> integration of a Tianmouc chip into an autonomous driving system<\/span><\/strong>, showcasing its abilities to enable <strong><span style=\"color: #ff0000;\">accurate, fast and robust perception,<\/span><\/strong> even in challenging corner cases on open roads. The primitive-based complementary sensing paradigm helps in overcoming fundamental limitations in developing vision systems for diverse open-world applications.&#8221;<\/p>\n<p style=\"text-align: justify;\">Zheyu Yang, Taoyi Wang, Yihan Lin, Yuguo Chen, Hui Zeng, Jing Pei, Jiazheng Wang, Xue Liu, Yichun Zhou, Jianqiang Zhang, Xin Wang, Xinhao Lv, Rong Zhao &amp; Luping Shi. <a href=\"https:\/\/www.nature.com\/articles\/s41586-024-07358-4\"><strong>A vision chip with complementary pathways for open-world sensing<\/strong><\/a>. Nature 629, 1027\u20131033 (2024). https:\/\/doi.org\/10.1038\/s41586-024-07358-4<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Zheyu Yang, Taoyi Wang, Yihan Lin, Yuguo Chen, Hui Zeng, Jing Pei, Jiazheng Wang, Xue Liu, Yichun Zhou, Jianqiang Zhang, Xin Wang, Xinhao Lv, Rong Zhao &amp; Luping Shi. A vision chip with complementary pathways for open-world sensing. Nature 629, 1027\u20131033 (2024). https:\/\/doi.org\/10.1038\/s41586-024-07358-4 Abstract &#8220;Image sensors face substantial challenges when dealing with dynamic, diverse and [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[74,7,4],"tags":[89,85,88,87,90,86],"class_list":["post-325","post","type-post","status-publish","format-standard","hentry","category-brain-inspired-robotics","category-neuromorphic-robotics","category-neuromorphic-sensing","tag-action-oriented-pathway","tag-brain-inspired-vision","tag-cognition-oriented-pathway","tag-complemetary-vision","tag-open-world-sensing","tag-tianmouc"],"_links":{"self":[{"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=\/wp\/v2\/posts\/325","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=325"}],"version-history":[{"count":1,"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=\/wp\/v2\/posts\/325\/revisions"}],"predecessor-version":[{"id":326,"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=\/wp\/v2\/posts\/325\/revisions\/326"}],"wp:attachment":[{"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=325"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=325"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/braininspiredrobotics.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=325"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}