In the Netherlands, IMEC-NL contributes to new micro-electronical and sensor technology solutions in the fields of health & vitality, photonics, energy & industry bring them forward into the digital age. By not only innovating on the building blocks, IMEC-NL also specifically aims to create differentiating and exploitable IP by connecting the various disciplines. The application domains are mainly datacom, automotive lidar and health. IMEC-NL is involved in PhotonDelta (€1.1 billion program) which offers an end-to-end supply chain that designs, develops, and manufactures photonic integrated circuits. IMEC-NL provides integrated photonics devices that are faster, cheaper, more powerful and more energy efficient. 

In NEHIL, IMEC-NL aims to design FMCW LiDAR for ranges of up to 100 meters, with pixel rates of up to 160,000 pixels per second, which is 8 times higher than state of art. With the Neuromorphic circuit for pointcloud data processing, our objective is to achieve a tenfold reduction in power consumption for object recognition and classification. We will focus on the design of more complete photonic systems, including electronics and algorithms. Heterogeneous integration plays a pivotal role in advancing FMCW LiDAR technology, particularly in achieving miniaturization and cost-effectiveness while integrating diverse functionalities. IMEC-NL’s integration process primarily adopts 2.5D integration techniques, such as Butt coupling and Evanescent coupling, to facilitate the coexistence of silicon and III/V chips within the same package. 

In NEHIL, IMEC-NL undertakes a dual role aimed at advancing FMCW LiDAR technology. Firstly, leveraging expertise in heterogeneous integration processes at IMEC-BE, IMEC-NL leads the architecture and implementation of a chip-scale LiDAR module. This module comprises essential components including FMCW pixel array, FMCW-grade on-chip laser, PIC driving ASIC, and associated software. Additionally, IMEC-NL leads efforts in constructing a demonstrator that combines the FMCW LiDAR module with the Neuromorphic computing circuits from DGIST, SNU, SU, TRT, CSIC, and IMEC-Gent. This combination enables enhanced sensing capabilities, facilitating high-sensitivity sensing in the pre-pointcloud stage and low-power object recognition and classification in the post-pointcloud stage. By conducting comprehensive system-level studies, we seek to validate the viability of Neuromorphic circuits as edge AI solutions for fusion sensors, including LiDAR, with a specific focus on applications such as autonomous driving. Through these endeavors, we aim to drive forward advancements in LiDAR technology, ultimately enhancing the safety and efficiency of autonomous systems. 

IMEC-BE considers the developments made in this project as a unique opportunity for extending and further developing its world-leading silicon photonics capabilities in Europe and beyond. The results generated within NEHIL will further extend IMEC-BE’s portfolio and strengthen its offering for next-generation (co-packaged) optical systems (such as LiDARs, optical transceivers operating at 200Gbps, and optical sensors). Achieving this will strengthen further IMEC-BE’s position in Europe and the rest of the world as one of the leading R&D organizations in the field. For LiDAR in particular, IMEC-BE considers this project to be a unique opportunity to further strengthen its position on next-generation solid-state systems that will be key for advancing autonomous vehicles. The results generated in this project on advanced packaging and module integration of low-loss waveguides will complement IMEC-BE’s existing IP on CMOS circuits. The outreach and exploitation of the project results will be done via IMEC-BE’s strong network of partners worldwide. Through its extensive ecosystem, as well as its prototyping and MPW offering in Silicon Photonics, IMEC-BE brings together different players in the process chain, from IC manufacturers to equipment, material, and software suppliers. In terms of direct contribution towards the European capability improvements, we see the NEHIL project as a direct link between the IMEC pilot line infrastructure, and many European based supply chain partners, including equipment and metrology suppliers providing access to the latest technology of heterogeneous integration processing, metrology and defectivity inspection capabilities. Europe holds a key position in automotive and this project through the novel chip-scale heterogeneously integrated LiDAR engine will contribute to maintaining that position in the coming years. As a research centre, IMEC acts as multiplier by disseminating their expertise to European and world-wide industry as well as to the academic world by educating next generation engineers and existing professional via high-impact journal publications, the IMEC academy and University relationships (such as joint research, lectures, and PhD students). 

The neuromorphic computing group at IMEC-BE-Gent has many years of experience in using nonconventional photonic hardware to tackle high-speed low-latency information processing in the optical domain. Applications include e.g. particle identification in flow cytometry, and problems in telecommunications, more specifically the undoing of nonlinear distortion in high-speed optical datalinks. This operation is typical performed in the digital electronic domain, at high cost in terms of chip area and power consumption. However, by replacing this with operation in the analogue photonic domain, important improvement in speed and power consumption are possible. This work has been supported by numerous EU projects, starting with an ERC grant. 

The knowledge created through the NEHIL project will enable IMEC-BE to have a better technology offer towards the industry in Europe and worldwide and will likely attract and bring new partnerships in the future to develop and commercialize new products, new processes, or new applications. 

Beyond its technological capabilities, IMEC (IMEC-NL and IMEC-BE) serves as a hub for innovation, collaboration and knowledge exchange. Successful spin-offs and recent examples of technology transfer in this subject by IMEC include BiFAST, SDNSquare, Caliopa. 

Thales is one of the major world players in civilian & professional electronics with over 77,000 employees with Global presence in 68 Countries, and 17.6 B€ Sales in 2022, Thales Research and Technology France (TRT) located in Palaiseau, near Paris, is the main multidisciplinary research unit of the Thales Group. TRT is participating in the preparation of Thales industrial future in strategic R&D fields, with 1 Nobel Prize, 3 European Research Council projects and also 80 French or European on-going collaborative research projects, With 240 permanent staff and on-going 40 PhD students, and 4,000 m² of cleanrooms and 200 characterization & process main equipment. TRT is dedicated to mastering increasingly sophisticated technologies, particularly those related to detection, analysis, or decision-making, in order to design and develop critical information systems. These innovative solutions serve clients in various domains such as secure communications, space systems, air traffic management, embedded electronics, large computer networks and administrations.  Project contribution will be carried out within the “Optical Signal Processing” and “Micro-Nano Physics” research groups. The Physics department has about 25 years of experience in different research topics in electro-optics (materials and devices), advanced laser sources, non-linear optics, optical signal processing and optoelectronics for microwave and sensors, such as FMCW LiDAR and distributed acoustic fiber sensors. 

The Optical signal Processing Laboratory within TRT has developed expertise in using optical techniques for advanced on-line, latency-free generation and conditioning of high-speed signals. In the last decade we have directed our efforts towards the photonic integration of these concepts. In parallel, the Micro and Nano Physics Laboratory within the Physics department of TRT has developed concepts for the miniaturization of photonic functions for all-optical (i.e. nonlinear) processing. All these ideas will converge in our activity in NEHIL, where TRT will aim at the experimental demonstration of recently proposed  architectures for compact photonic processors. They will be used for analog computing on a class of signals of high relevance with a drastic improvement in energy efficiency.  

To this aim TRT has several laboratories dedicated to microwave photonics, including microwave equipment of for signal generation, detection and monitoring; probe stations for electrical connections, positioning stages for nanometric accuracy, a system for accurate  alignment and assembly of heterogeneous integration of photonic circuits. For simulations, we have recently installed a high-performance computing cluster with >1000 CPU cores and several Nvidia A100 units. 

The Spanish Research Council (CSIC), the largest public research institution in Spain, holds a prominent position within the European Research Area (ERA). The CSIC researchers participating in this project are affiliated with the Institute for Cross-disciplinary Physics and Complex Systems (IFISC), a joint entity established by CSIC and the University of the Balearic Islands (UIB). Notably, IFISC has been distinguished as a two-time recipient of the Maria de Maeztu excellence program, Spain’s highest recognition in research. The CSIC researchers involved in the NEHIL project focus on photonics and dynamical systems research. To support their scientific activities, the CSIC group at IFISC has access to powerful research infrastructure, including a High-Performance Computing (HPC) cluster featuring 960 cores, 12TB of memory, and 25Gb/s low latency communications. Additionally, the group operates a state-of-the-art nonlinear photonics laboratory, equipped for ultra-fast signal generation and acquisition. 

In NEHIL, CSIC will explore strategies for increasing robustness to hardware failures in neuromorphic chips, which have potential applications in real-world tasks, such as LIDAR object recognition and telecom signal recovery.
CSIC researchers at IFISC have led several cutting-edge projects in the field of neuro-inspired photonic information processing. Over the past decade, the IFISC team has been at the forefront of this field, driving innovation through the development of key concepts and their real-world implementation. Notably, they successfully coordinated the FET-Open project PHOCUS, which focused on the photonic implementation of reservoir computing. Currently, IFISC researchers are advancing the field further as participants in the H2020 FET project “Adaptive Optical Dendrites (ADOPD)”, which aims to develop ultrafast fibre-optical computing units inspired by brain-derived dendritic computing concepts. This project promises to pave the way for next-generation neuromorphic computing devices with significantly reduced energy consumption. Additionally, IFISC researchers are contributing to the European doctoral training network PostDigital, focusing on reservoir and neuromorphic computing topics. 

At Korean level, NEHIL’s consortium involves a group of three experienced Universities (DGIST, SNU and CSIC), with cutting edge facilities to carry out the proposed research activities.  

In addition, the KR team gathers two Letter-of-Intent from two industrial partners (Rebellions and Telechips), that will bring knowledge-based industry and ensure the commercial viability of the research results. 

The project management of the ‘joint agreement’ plans to facilitate the interaction of this EU-KR ecosystem actively, making sure that all partners interact and collaborate to go beyond state-of-art across the value chain. 

DGIST was established in 2003 with a focus on interdisciplinary education and research, featuring seven departments: Electrical Engineering and Computer Science, Robotics Engineering and Mechatronics, Chemistry and Physics, Neuroscience, New Biology, Energy Engineering, and an interdisciplinary program. DGIST uniquely offers flexible undergraduate and graduate programs emphasizing interdisciplinary study, with no barriers between departments. Although DGIST was founded in 2003, It is one of the fastest growing universities in the world (ranked as 7 th in Nature Index Top 25 rising universities in 2019) and it is ranked as 2nd place in Korea for number of papers and paper citation index per faculty member(Asia University Rankings 2023) 

DGIST features advanced research facilities and resources dedicated to semiconductor technology, integrated circuits, and artificial intelligence. Among its extensive facilities, DGIST Central Equipment center has established the second standard semiconductor process facility in Korea (costing 40 million USD), providing a research environment for AI semiconductor and system development (including development of neuromorphic devices and circuit design for AI semiconductor applications). Semiconductor fabrication facility within the center has top-tier cleanroom facilities, including a dedicated 4,431 m² building with a 1,596 m² device cleanroom and a 1,403 m² materials analysis room. It possesses 84 pieces of equipment including a Laser Writer, EBL, I-line Stepper, Ion Implanter, CDSEM and offers a 6-inch MOSFET , 0.5 μm CMOS process for SoC design, as well as services for compound semiconductor and MEMS processes for AI semiconductor sensors. They also have their own 0.5 μm CMOS PDK and manufacturing technology for 6-inch CMOS, providing full-custom design education for digital and analog circuits for AI semiconductor circuit designers, allowing for device integration and the development of new devices such as neuromorphic devices at the same location. 

DGIST also offers top-notch AI computing infrastructure, including the iREMB supercomputer, ranked 454th globally in 2017, and a fifth machine built in 2020 at a total cost of 10 billion KRW, for education and research in AI semiconductor and system design.  

In this project, the focus of research is developing FMCW LiDAR based neuromorphic computing system for perception tasks of autonomous vehicle/unmanned aerial vehicle(UAV). The research is divided into four sub research areas.  

First sub research area is developing hardware-aware hybrid spiking neural network(SNN)/artificial neural network(ANN) model that leverages benefits of FMCW LiDAR. By applying hybrid SNN/ANN on a deep learning model for perception tasks and hardware-aware efficient AI techniques such as quantization and sparsity, we expect significant power saving when the model is mapped to compute-in-memory hardware that utilizes neuromorphic devices.  

Second sub research area is security-aware ferroelectric FET based neuromorphic compute in memory accelerator design. The team will research on ferroelectric FET suitable for compute-in-memory operation as well as secure and efficient compute-in-memory accelerator. In addition, the accelerator will also support spiking neural networks with Leaky Integrate Fire neuron circuit design. This is targeting the acceleration of SNN in hybrid SNN/ANN deep learning model mentioned above. In this research, we will research LIF neuron integrated compute-in-memory circuit and analog spiking neural net with LIF neurons to find out which configuration yields better acceleration of SNN in terms of latency and power. 

Third sub research area is RF communication and high-speed interface circuits for FMCW LiDAR and sensors & processors. Since perception tasks need co-ordination of FMCW LiDAR and multiple other sensors with processors, we need reliable, high bandwidth and low latency communication circuits. 

Seoul National University (SNU) is one of the most prestigious national universities in South Korea, established in 1946. In total, it has about 16,000 undergraduate and 12,000 graduate students and more than 2,000 full-time faculty members. SNU stands as a pinnacle of academic excellence not only in South Korea but also globally. Renowned for its commitment to rigorous research across a multitude of disciplines, SNU consistently ranks among the world’s top research universities. From groundbreaking discoveries in biotechnology and medicine to pioneering advancements in artificial intelligence and robotics, SNU’s research endeavors have a profound impact on society and contribute to addressing some of the world’s most pressing challenges. 

SNU boasts state-of-the-art research facilities and resources that empower semiconductor technology, integrated circuits, and artificial intelligence areas. Among many research facilities, Inter-university Semiconductor Research Center (ISRC) at Seoul National University was established in 1988 as the first research facility in Korea for joint utilization of research equipment, aiming at semiconductor basic research and advanced education. For over 30 years, the ISRC has provided joint utilization of equipment, including over 23,000 cases annually, involving a total of over 10,700 students, including over 4,900 personnel from companies, and possesses the highest level of educational and research capabilities and operational know-how for research facilities. The ISRC has established several standard fabrication processes in (1) 0.5-µm CMOS processes, (2) 0.5-µm poly TFTs, and (3) 0.25-µm CMOS processes, available for research in this project. 

In NEHIL, we will fully utilize the ISRC facilities for the successful implementation of the heterojunction integration neuromorphic devices. For the proposed topics in neuromorphic FeFET devices and their application in neuromorphic computing, established processes such as 0.25 µm CMOS structures (2 metals) and Back-end processing such as STI (shallow trench isolation), CMP and metal processes will be used to successfully integrate the devices. 

Sogang University was established by the Society of Jesus to provide education based on Catholic beliefs and inspired by the Jesuit philosophy of education, in conformity with the Korean tradition of education. Sogang University aims to educate the whole person with love and faith, respect the values of human dignity, and encourage the pursuit of learning with a sincere quest for truth. Through this education, Sogang University nurtures the talents who will devote their lives to the development of a humanistic culture and community. The number of full-time faculty members is more than 400, supporting and advising about 8,100 and 3,700 students in undergraduate and graduate courses, respectively. 

Sogang Univerisity has been dedicated to cultivating true talents who can contribute to the development of human culture and the advancement of humanity under the vision and progress of being a specialized top-ranked engineering college within the top 100 in the world and the best in Korea, leading Sogang on the global stage. Building on externally recognized research capabilities, the college conducts various national projects such as BK21 FOUR and has played a pivotal role in the industrial development of South Korea by establishing strong ties with excellent domestic companies through an industry-academia cooperation system. It has attracted exchange students from various countries around the world including the United States, Europe, and Asia, and has established joint research centers, advancing its globalization efforts. 

In this research, we research a low-power integrate-and-fire (IF) neuron for implementing SNN to process images from an FMCW-based LiDAR sensor. An analog counter consisting of two capacitors and four switches provides a linear counting nature, which converts spike signals in the time domain to membrane potential in the voltage domain. Two membrane potentials in positive and negative neurons are compared and fired when the difference is larger than a threshold level. The proposed IF neuron performs the multiply-and-addition (MAC) operation essential for SNN by utilizing the spike signal and weighting coefficient of the surrounding 3×3 computing-inmemory (CIM) macro. Each basic unit has a counter and register for rate coding, and a discharging 

capacitor for the analog counter. The integrating capacitor is shared with other units to combine the surrounding spike signal values. During MAC operation, the counter is reorganized into a register to store the weight. If the spike value stored in the register is 1, the number of pulses corresponding to the weight value is generated, which operates the analog counter and generates a membrane potential. After firing occurs, another spike is generated and transmitted to the SNN accelerator with the next layer.