OpenHarmony 源碼解析之多模輸入子系統(事件派發流程)
簡介
多模輸入系統主要用于接收按鍵,觸摸等輸入事件,并且會對這些原始輸入事件進行處理,之后再對這些事件進行派發。同時多模輸入系統還提供了注入事件的接口,應用可以通過調用這個接口產生輸入事件,然后將該輸入事件注入到輸入系統中進行處理。

輸入系統框架

多模輸入系統主要是由InputManagerService, InputEventHub, InputEventDistributer來負責處理的。InputManagerService會啟動InputEventHub,并且會通過創建子線程的方式來創建InputEventDistributer。當底層傳來按鍵或觸摸事件的時候,InputEventHub就會進行讀取,并且會對這些原始的輸入事件進行處理,處理完后會交給InputEventDistributer進行派發。InputEventDistributer又會通過InputEventClientProxy進行IPC交互的方式發給應用端。
多模輸入系統事件派發流程
事件派發流程圖

源碼分析
下面就對多模輸入系統事件派發流程的源碼進行分析。
InputManagerService
\foundation\graphic\wms\services\wms\wms.cpp
- int main()
- {
- DEBUG_PERFORMANCE_REGISTER_SIG();
- OHOS::HiFbdevInit();
- OHOS::GfxEngines::GetInstance()->InitDriver();
- HOS_SystemInit();
- OHOS::InputManagerService::GetInstance()->Run();
- while (1) {
- DEBUG_PERFORMANCE_PRINT_RESULT();
- OHOS::LiteWM::GetInstance()->MainTaskHandler();
- usleep(WMS_MAIN_TASK_PERIOD_IN_US);
- }
- }
InputManagerService的啟動是在WMS的main函數中通過InputManagerService::GetInstance()->Run()執行的。
\foundation\graphic\wms\services\ims\input_manager_service.cpp
- void InputManagerService::Run()
- {
- hub_ = InputEventHub::GetInstance();
- hub_->RegisterReadCallback(ReadCallback);
- hub_->SetUp();
- distributerThreadCreated_ = pthread_create(&distributerThread_, nullptr, Distribute, nullptr);
- if (!distributerThreadCreated_) {
- pthread_detach(distributerThread_);
- }
- }
在InputManagerService::Run()中首先會創建InputEventHub的對象并通過RegisterReadCallback來注冊InputEventHub的回調,然后通過SetUp來啟動InputEventHub, InputEventHub主要是用于對底層原始輸入事件的讀取和處理,該函數的最后會創建distributerThread子線程,用于對輸入事件的派發。
InputEventHub
\foundation\graphic\wms\services\ims\input_event_hub.cpp
- void InputEventHub::SetUp()
- {
- int32_t ret = GetInputInterface(&inputInterface_);
- if (ret != INPUT_SUCCESS) {
- GRAPHIC_LOGE("get input driver interface failed!");
- return;
- }
- uint8_t num = ScanInputDevice();
- if (num == 0) {
- GRAPHIC_LOGE("There is no device!");
- return;
- }
- for (uint8_t i = 0; i < num; i++) {
- if (inputInterface_ == nullptr || inputInterface_->iInputManager == nullptr) {
- GRAPHIC_LOGE("input interface or input manager is nullptr, open device failed!");
- return;
- }
- ret = inputInterface_->iInputManager->OpenInputDevice(mountDevIndex_[i]);
- if (ret == INPUT_SUCCESS && inputInterface_->iInputReporter != nullptr) {
- callback_.EventPkgCallback = EventCallback;
- ret = inputInterface_->iInputReporter->RegisterReportCallback(mountDevIndex_[i], &callback_);
- if (ret != INPUT_SUCCESS) {
- GRAPHIC_LOGE("device dose not exist, can't register callback to it!");
- return;
- }
- openDev_ = openDev_ | (1 << i);
- }
- }
- }
在這個函數中InputEventHub主要的工作就是通過調用驅動層的OpenInputDevice來打開輸入設備,并且會將EventCallback的回調函數通過驅動層的RegisterReportCallback進行注冊。當底層有事件傳遞上來,EventCallback就會被調用。OpenInputDevice和RegisterReportCallback具體實現分別是在drivers/peripheral/input/hal/src/input_manager.c和drivers/peripheral/input/hal/src/input_reporter.c中。
\foundation\graphic\wms\services\ims\input_event_hub.cpp
- void InputEventHub::EventCallback(const EventPackage **pkgs, uint32_t count, uint32_t devIndex)
- {
- if (pkgs == nullptr || readCallback_ == nullptr || count == 0) {
- return;
- }
- RawEvent& data = InputEventHub::GetInstance()->data_;
- for (uint32_t i = 0; i < count; i++) {
- if (pkgs[i]->type == EV_REL) {
- data.type = InputDevType::INDEV_TYPE_MOUSE;
- if (pkgs[i]->code == REL_X)
- data.x += pkgs[i]->value;
- else if (pkgs[i]->code == REL_Y)
- data.y += pkgs[i]->value;
- } else if (pkgs[i]->type == EV_ABS) {
- data.type = InputDevType::INDEV_TYPE_TOUCH;
- if (pkgs[i]->code == ABS_MT_POSITION_X)
- data.x = pkgs[i]->value;
- else if (pkgs[i]->code == ABS_MT_POSITION_Y)
- data.y = pkgs[i]->value;
- } else if (pkgs[i]->type == EV_KEY) {
- if (pkgs[i]->code == BTN_MOUSE || pkgs[i]->code == BTN_TOUCH) {
- if (pkgs[i]->value == 0)
- data.state = 0;
- else if (pkgs[i]->value == 1)
- data.state = 1;
- }
- } else if (pkgs[i]->type == EV_SYN) {
- if (pkgs[i]->code == SYN_REPORT) {
- break;
- }
- }
- }
- readCallback_(&data);
- }
當底層有輸入事件上來的話,EventCallback就會被調用,在這個函數里會通過EventPackage->type來判斷輸入事件的類型,其中
EV_REL是相對坐標的輸入事件,比如軌跡球,鼠標事件
EV_ABS是絕對坐標的輸入事件,比如觸屏觸摸事件
EV_KEY是按鍵輸入事件,比如設備上的物理按鍵的點擊事件
EV_SYN是Motion的一系列動作結束標志位
如果是鼠標事件,會將相對坐標值放入到data.x和data.y中,如果是觸屏觸摸事件,會將在觸屏上觸摸的坐標位置放入到data.x和data.y中,如果是按鍵事件會將按鍵的點擊狀態放入到data.state中。
處理完輸入事件后,會將數據放入到data中,并通過readCallback傳給InputManagerService進行處理,之后就會調用InputManagerService::ReadCallback。
\foundation\graphic\wms\services\ims\input_manager_service.cpp
- void InputManagerService::ReadCallback(const RawEvent* event)
- {
- if (event == nullptr) {
- return;
- }
- pthread_mutex_lock(&lock_);
- while (eventQueue_.size() == MAX_EVENT_SIZE) {
- pthread_cond_wait(&nonFull_, &lock_);
- }
- // push events into queue
- eventQueue_.push(event[0]);
- pthread_mutex_unlock(&lock_);
- pthread_cond_signal(&nonEmpty_);
- }
- void* InputManagerService::Distribute(void* args)
- {
- GRAPHIC_LOGI("InputManagerService::Distribute Ready to read distribute!");
- while (true) {
- pthread_mutex_lock(&lock_);
- while (eventQueue_.size() == 0) {
- pthread_cond_wait(&nonEmpty_, &lock_);
- }
- // pop events from queue
- RawEvent events[MAX_INPUT_DEVICE_NUM];
- int32_t len = (eventQueue_.size() > MAX_EVENT_SIZE) ? MAX_EVENT_SIZE : eventQueue_.size();
- for (int32_t i = 0; i < len; i++) {
- events[i] = eventQueue_.front();
- eventQueue_.pop();
- }
- distributer_.Distribute(events, len);
- pthread_mutex_unlock(&lock_);
- pthread_cond_signal(&nonFull_);
- }
- return nullptr;
- }
ReadCallback這個函數首先會判斷eventQueue這個事件隊列里事件數量是否達到最大數量,如果達到最大數量該線程就一直等待,否則就會把該事件放到eventQueue這個事件隊列里,并且同時也會發出nonEmpty的signal, 來讓Distribute中的線程停止等待。
Distribute函數中,當eventQueue隊列里沒有事件的時候,就會一直等待,當有事件來的時候就會停止線程等待,然后會遍歷整個eventQueue這個隊列,把每個事件獲取出來后放入到events這個數組中,并做為參數放入到InputEventDistributer::Distribute中進行事件的派發。
InputEventDistributer
\foundation\graphic\wms\services\ims\input_event_distributer.cpp
- void InputEventDistributer::Distribute(const RawEvent* events, int32_t size)
- {
- for (int32_t i = 0; i < size; i++) {
- for (auto listener : rawEventListeners_) {
- if (listener != nullptr) {
- listener->OnRawEvent(events[i]);
- }
- }
- }
- }
這個函數比較簡單,主要就是遍歷所有的InputEventClientProxy, 并且調用各自的onRawEvent進行實際的派發工作。
InputEventClientProxy
\foundation\graphic\wms\services\ims\input_event_client_proxy.cpp
- void InputEventClientProxy::OnRawEvent(const RawEvent& event)
- {
- IpcIo io;
- uint8_t tmpData[IMS_DEFAULT_IPC_SIZE];
- IpcIoInit(&io, tmpData, IMS_DEFAULT_IPC_SIZE, 1);
- IpcIoPushFlatObj(&io, static_cast<const void*>(&event), sizeof(RawEvent));
- pthread_mutex_lock(&lock_);
- std::map<pid_t, ClientInfo>::iterator it;
- for (it = clientInfoMap_.begin(); it != clientInfoMap_.end(); it++) {
- if (it->second.alwaysInvoke || (event.state != lastState_)) {
- SendRequest(nullptr, it->second.svc, 0, &io, nullptr, LITEIPC_FLAG_ONEWAY, nullptr);
- }
- }
- lastState_ = event.state;
- pthread_mutex_unlock(&lock_);
- }
這個函數主要就是通過ipc的交互方式把輸入事件傳給應用端。
到此整個多模輸入系統的事件派發流程就結束了。
多模輸入系統接口說明
模塊
- /foundation/multimodalinput/input
- ├── common # 公共代碼
- ├── interfaces # 對外接口存放目錄
- │ └── native # 對外native層接口存放目錄
- │ └── innerkits # 對系統內部子系統提供native層接口存放目錄
- ├── service # 服務框架代碼
- ├── sa_profile # 服務啟動配置文件
- ├── uinput # 輸入事件注入模塊
通過每個目錄下的.gn文件可以看到每個目錄下的模塊都對應動態庫
\interfaces\native\innerkits\event下的文件編出來的是mmi_event.so
\interfaces\native\innerkits\napi 下的文件編出來的是injecteventhandler.so
\interfaces\native\innerkits\proxy 下的文件編出來的是libmultimodalinput_proxy.so
\service 下的文件編出來的是libmultimodalinput_service.so
\uinput 下的文件編出來的是mmi_uinject.so
接口
多模輸入目前提供的接口為事件注入接口,該接口目前僅對系統應用開放。
JS接口
InJectEventHandler是處理注入事件類。
\applications\standard\systemui\navigationBar\src\main\js\default\pages\backKey\backKey.js
- export default {
- /**
- * User start touching the back button
- */
- backTouchStart() {
- mLog.showInfo(TAG, `back touch start`);
- res = input.injectEventSync({
- isPressed: true,
- keyCode: 2,
- keyDownDuration: 1
- });
- mLog.showInfo(TAG, `injectEventHandler injectEventSync down res: ${res}`);
- },
- /**
- * User stop touching the back button
- * Trigger "Back" event
- */
- backTouchEnd() {
- mLog.showInfo(TAG, `back touch end and injectEventHandler injectEventSync`);
- res = input.injectEventSync({
- isPressed: false,
- keyCode: 2,
- keyDownDuration: 1
- });
- mLog.showInfo(TAG, `injectEventHandler injectEventSync up res: ${res}`);
- }
- }
可以從openharmony systemui的navigationbar的源碼中看到, 當點擊navigationbar的back鍵的時候,就會調用js的接口函數injectEventSync,并傳入三個參數,其中
isPress: 按鍵的狀態,true表示down, false表示up
keyCode:鍵值碼,2表示back事件
keyDownDuration:按鍵按下到抬起之間的時長,單位ms,1表示1ms
C++接口
系統內部接口
在\interfaces\native\innerkits\events\include下的頭文件都定義了各自對內部系統調用的口。
KeyEvent的主要接口
KeyBoardEvent的主要接口
ManipulationEvent的主要接口
MmiPoint的主要接口
MouseEvent的主要接口
MultimodalEvent的主要接口
StylusEvent的主要接口
TouchEvent的主要接口
InjectEvent的實現邏輯
\foundation\multimodalinput\input\interfaces\native\innerkits\napi\src\key_event_handler.cpp
- static napi_value InjectEventSync(napi_env env, napi_callback_info info)
- {
- size_t argc = 2;
- napi_value args[2] = { 0 };
- napi_value thisArg = nullptr;
- void* data = nullptr;
- NAPI_CALL(env, napi_get_cb_info(env, info, &argc, args, &thisArg, &data));
- napi_value eventObject = args[0];
- int32_t ret = IsMatchType(eventObject, napi_object, env);
- if (ret) {
- return GetNapiInt32_t(ret, env);
- }
- napi_value isPressed, keyCode, keyDownDuration;
- napi_get_named_property(env, eventObject, "isPressed", &isPressed);
- napi_get_named_property(env, eventObject, "keyDownDuration", &keyDownDuration);
- napi_get_named_property(env, eventObject, "keyCode", &keyCode);
- if (IsMatchType(isPressed, napi_boolean, env) || IsMatchType(keyCode, napi_number, env)
- || IsMatchType(keyDownDuration, napi_number, env)) {
- return GetNapiInt32_t(-1, env);
- }
- OHOS::KeyProperty keyProperty = {
- .isPressed = GetCppBool(isPressed, env),
- .keyCode = GetCppInt32_t(keyCode, env),
- .keyDownDuration = GetCppInt32_t(keyDownDuration, env),
- };
- OHOS::MultimodalProperty multimodalProperty {
- .highLevelEvent = 1,
- .uuid = "11111",
- .sourceType = 1,
- .occurredTime = 1,
- .deviceId = "11111",
- .inputDeviceId = 1,
- .isHighLevelEvent = true,
- };
- OHOS::sptr<OHOS::KeyEvent> event = new OHOS::KeyEvent();
- if (!event) {
- return GetNapiInt32_t(-1, env);
- }
- event->Initialize(multimodalProperty, keyProperty);
- std::shared_ptr<OHOS::InjectManager> injectManager = OHOS::InjectManager::GetInstance();
- bool isSucceed = injectManager->InjectEvent(event);
- if (!isSucceed) {
- return GetNapiInt32_t(-1, env);
- }
- return GetNapiInt32_t(0, env);
- }
在key_event_handler.cpp中實現了InjectEventSync這個接口,通過NAPI獲得應用端的isPressed,KeyDownDuration,KeyCode這三個數值,并將這三個參數放入到KeyProperty這個結構體中。然后調用KeyEvent的Initialize,將KeyProperty封裝到KeyEvent中,最后再調用InjectManager的InjectEvent。
\foundation\multimodalinput\input\interfaces\native\innerkits\proxy\src\inject_manager.cpp
- bool InjectManager::InjectEvent(const sptr<MultimodalEvent> event)
- {
- std::lock_guard<std::mutex> guard(lock_);
- if (!multimodalInputService_) {
- return false;
- }
- int32_t result = multimodalInputService_->InjectEvent(event);
- if (result == 0) {
- return true;
- }
- MMI_LOGI("inject failed");
- return false;
- }
foundation\multimodalinput\input\interfaces\native\innerkits\proxy\include\inject_manager.h
- sptr<IMultimodalInputService> multimodalInputService_{nullptr};
multimodalInputService_->InjectEvent其實是一個IPC進程間調用,這會調用到客戶端的MultimodalInputServiceProxy的InjectEvent。
foundation\multimodalinput\input\interfaces\native\innerkits\proxy\src\multimodal_input_service_proxy.cpp
- int32_t MultimodalInputServiceProxy::InjectEvent(const sptr<MultimodalEvent> &event)
- {
- MessageParcel data;
- MessageParcel reply;
- MessageOption option(MessageOption::TF_ASYNC);
- if (!data.WriteInterfaceToken(MultimodalInputServiceProxy::GetDescriptor())) {
- HiLog::Error(LABEL, "write descriptor fail");
- return ERR_INVALID_VALUE;
- }
- if (!data.WriteInt32(MultimodalEvent::KEYBOARD)) {
- HiLog::Error(LABEL, "write descriptor fail");
- return ERR_INVALID_VALUE;
- }
- if (!data.WriteParcelable(event)) {
- HiLog::Error(LABEL, "inject event fail, write event error");
- return ERR_INVALID_VALUE;
- }
- int error = Remote()->SendRequest(INJECT_EVENT, data, reply, option);
- if (error != ERR_NONE) {
- HiLog::Error(LABEL, "inject event fail, error: %{public}d", error);
- }
- return error;
- }
在MultimodalInputServiceProxy::InjectEvent會通過SendRequest向服務端MultimodalInputServiceStub發送數據。
foundation\multimodalinput\input\service\src\multimodal_input_service_stub.cpp
- int MultimodalInputServiceStub::OnRemoteRequest(
- uint32_t code, MessageParcel &data, MessageParcel &reply, MessageOption &option)
- {
- MMI_LOGD("OnReceived, cmd = %{public}u", code);
- if (!IsPermissionValid()) {
- MMI_LOGE("calling app not acquired multimodal permission");
- return MMI_PERMISSION_ERR;
- }
- std::u16string myDescripter = MultimodalInputServiceStub::GetDescriptor();
- std::u16string remoteDescripter = data.ReadInterfaceToken();
- if (myDescripter != remoteDescripter) {
- MMI_LOGE("descriptor checked fail");
- return MMI_BAD_TYPE;
- }
- switch (code) {
- case INJECT_EVENT: {
- int32_t type = data.ReadInt32();
- if (type == MultimodalEvent::KEYBOARD) {
- sptr<MultimodalEvent> event = data.ReadParcelable<KeyEvent>();
- return InjectEvent(event);
- }
- MMI_LOGE("recv bad type %{public}d", type);
- return MMI_BAD_TYPE;
- }
- default: {
- MMI_LOGE("default case, need check");
- return IPCObjectStub::OnRemoteRequest(code, data, reply, option);
- }
- }
- }
通過sendRequest將數據發送之后,服務端的MultimodalInputServiceStub的OnRemoteRequest就會被調用,最終會調用MultimodaInputService的InjectEvent。
\foundation\multimodalinput\input\service\src\multimodal_input_service.cpp
- int32_t MultimodalInputService::InjectEvent(const sptr<MultimodalEvent> &event)
- {
- KeyEvent *eventPtr = reinterpret_cast<KeyEvent*>(event.GetRefPtr());
- int keycode = eventPtr->GetKeyCode();
- int state = 0;
- if (eventPtr->IsKeyDown()) {
- state = 1;
- } else {
- state = 0;
- }
- MMIS::KeyboardInject &inject = OHOS::MMIS::KeyboardInject::GetInstance();
- MMI_LOGD("InjectEvent keycode %{public}d, state %{public}d", keycode, state);
- inject.InjectKeyEvent(keycode, state);
- return 0;
- }
MultimodaInputService的InjectEvent實際上會調用KeyboardInject的InjectKeyEvent,從函數的實現來看,目前只使用了KeyboardInject,也就是說目前只支持鍵盤事件的注入。
\foundation\multimodalinput\input\uinput\keyboard_inject.cpp
- void KeyboardInject::InjectKeyEvent(uint16_t code, uint32_t value) const
- {
- std::lock_guard<std::mutex> keyboardLock(mutex_);
- auto it = keyCodeMap_.find(code);
- if (it == keyCodeMap_.end()) {
- return;
- }
- InjectInputEvent injectInputEvent = {injectThread_->KEYBOARD_DEVICE_ID, EV_KEY, it->second, value};
- injectThread_->WaitFunc(injectInputEvent);
- InjectInputEvent injectInputSync = {injectThread_->KEYBOARD_DEVICE_ID, EV_SYN, SYN_REPORT, 0};
- injectThread_->WaitFunc(injectInputSync);
- }
在InjectKeyEvent中會通過InjectInputEvent的WaitFunc將注入事件繼續向下注入。
\foundation\multimodalinput\input\uinput\inject_thread.cpp
- void InjectThread::InjectFunc() const
- {
- std::unique_lock<std::mutex> uniqueLock(mutex_);
- while (true) {
- conditionVariable_.wait(uniqueLock);
- while (injectQueue_.size() > 0) {
- if (injectQueue_[0].deviceId == TOUCH_SCREEN_DEVICE_ID) {
- g_pTouchScreen->EmitEvent(injectQueue_[0].type, injectQueue_[0].code, injectQueue_[0].value);
- } else if (injectQueue_[0].deviceId == KEYBOARD_DEVICE_ID) {
- g_pKeyboard->EmitEvent(injectQueue_[0].type, injectQueue_[0].code, injectQueue_[0].value);
- }
- injectQueue_.erase(injectQueue_.begin());
- }
- }
- }
- void InjectThread::WaitFunc(InjectInputEvent injectInputEvent) const
- {
- std::lock_guard<std::mutex> lockGuard(mutex_);
- injectQueue_.push_back(injectInputEvent);
- conditionVariable_.notify_one();
- }
在WaitFunc中會將injectInputEvent放入到injectQueue這個隊列中,這個隊列是用來存放injectInputEvent的,并且通過notify_one來喚醒InjectThread,由于目前只支持鍵盤類型事件的注入,所有只會調用g_pKeyboard->EmitEven(),g_pKeyboard是VirtualKeyboard的對象,VirtualKeyboard又繼承自VirtualDevice,因此最終會調用VirtualKeyboard的EmitEvent。
foundation\multimodalinput\input\uinput\virtual_device.cpp
- bool VirtualDevice::EmitEvent(uint16_t type, uint16_t code, uint32_t value) const
- {
- struct input_event event {};
- event.type = type;
- event.code = code;
- event.value = value;
- #ifndef __MUSL__
- gettimeofday(&event.time, NULL);
- #endif
- if (write(fd_, &event, sizeof(event)) < static_cast<ssize_t>(sizeof(event))) {
- HiLog::Error(LABEL, "Event write failed %{public}s aborting", __func__);
- return false;
- }
- return true;
- }
- bool VirtualDevice::SetUp()
- {
- fd_ = open("/dev/uinput", O_WRONLY | O_NONBLOCK);
在該函數中會將這個注入事件寫入到文件描述符為fd_的設備文件中,從SetUp的函數中可以看出實際是寫入到/dev/uinput這個設備文件中。
到此多模輸入系統接口的介紹以及InjectEvent整個注入事件的流程就結束了。
總結
通過本文的學習可以了解多模輸入系統事件派發的流程,以及多模輸入系統的接口和注入事件的流程,結合以上的源碼分析會對多模輸入子系統會有更深入的理解。