mirror of
https://github.com/emilk/egui.git
synced 2026-07-19 04:58:54 +00:00
Umaĵo
5bf62ca4b3
<!-- Please read the "Making a PR" section of [`CONTRIBUTING.md`](https://github.com/emilk/egui/blob/main/CONTRIBUTING.md) before opening a Pull Request! * Keep your PR:s small and focused. * The PR title is what ends up in the changelog, so make it descriptive! * If applicable, add a screenshot or gif. * If it is a non-trivial addition, consider adding a demo for it to `egui_demo_lib`, or a new example. * Do NOT open PR:s from your `master` branch, as that makes it hard for maintainers to test and add commits to your PR. * Remember to run `cargo fmt` and `cargo clippy`. * Open the PR as a draft until you have self-reviewed it and run `./scripts/check.sh`. * When you have addressed a PR comment, mark it as resolved. Please be patient! I will review your PR, but my time is limited! --> * Closes N/A * [x] I have followed the instructions in the PR template This PR adds visual support for IME composition, including the cursor and conversion segment. These visuals works (mostly) well on native platforms (`egui-winit`). On the web (`eframe/web`), support is limited by browser capabilities: Chromium works well, Firefox shows partial improvement, and Safari remains subpar. > [!NOTE] > > For `eframe` on Windows, this feature is currently gated behind the `windows_new_ime_composition_visuals` feature flag. ## Details We extend `egui::ImeEvent::Preedit(String)` to `egui::ImeEvent::Preedit { text: String, active_range_chars: Option<std::ops::Range<usize>> }`. The new `active_range_chars` field enables rendering of: - the cursor (when the range is empty), and - the conversion segment (when the range is non-empty) in IME composition. In `egui-winit`, we now use the range provided by `winit::event::Ime::Preedit` instead of ignoring it. In `eframe/web`, we derive the range from `selectionStart` and `selectionEnd` on the text agent. This mapping is fully accurate only in Chromium, but represents the best available approach for now. ## Demonstrations ### Chinese IMEs (Shuangpin) We can see where the cursor is now. | What | With this PR | Without this PR | |-|-|-| | macOS builtin |<video src=https://github.com/user-attachments/assets/487c7e7c-ef6d-4a86-8dbc-8c71871b4470 />|<video src=https://github.com/user-attachments/assets/49bd5a60-4b90-4e4a-99e0-cd01d3f7030c />| | macOS builtin (light)|<video src=https://github.com/user-attachments/assets/e84546f6-947b-4cea-a87e-fda903f49164 />|——| | Windows builtin |<video src=https://github.com/user-attachments/assets/fd331884-1f0c-4822-a99e-8140aed54815 />|——| | Wayland iBus Intelligent Pinyin |<video src=https://github.com/user-attachments/assets/b6796c75-1c4e-45e5-b43a-5d8dea320485 />|——| | Chromium (Chrome) macOS | Identical to `macOS builtin`. |——| | Safari macOS | We can now differentiate between IME composition and text selection, but we still can't tell where the cursor is. |——| | Firefox (Zen) macOS | Identical to `macOS builtin`. |——| ### Japanese IMEs We can see where the conversion segment is now. | What | With this PR | Without this PR | |-|-|-| | macOS builtin |<video src=https://github.com/user-attachments/assets/f2994cd4-a966-4ff0-9590-d263c202ec76 />|<video src=https://github.com/user-attachments/assets/7cf5ff35-003d-4f60-8fbf-15c725c3ecb9 />| | macOS builtin (light)|<video src=https://github.com/user-attachments/assets/6f562bdd-12fc-4486-b37b-8fcf11643295 />|——| | Windows builtin |<video src=https://github.com/user-attachments/assets/f0905659-5335-4034-abda-c25cf8f2fd57 />|——| | Wayland iBus Anthy |<video src=https://github.com/user-attachments/assets/94cd3a24-3158-4d79-ae02-d9b30fdfa738 />|——| | Chromium (Chrome) macOS | Identical to `macOS builtin`. |——| | Safari macOS | We can now differentiate between IME composition and text selection, but we still can't tell where the conversion segment is. |——| | Firefox (Zen) macOS | (Limited improvement.) <video src=https://github.com/user-attachments/assets/3daf9b63-6e75-467b-8515-31c2a44adf61 /> |——| ### Korean IMEs We can clearly tell whether we are in composition (in contrast to selection) now. | What | With this PR | Without this PR | |-|-|-| |macOS builtin|<video src=https://github.com/user-attachments/assets/73ca28c7-22a0-493f-8f4d-c6e59a2dec54 />|<video src=https://github.com/user-attachments/assets/f582de7d-7ec0-48fe-910f-0139ef1620d3 />| |macOS builtin (light)|<video src=https://github.com/user-attachments/assets/269f03bd-6f95-498b-9fb1-1adcb043c738 />|——| | Windows builtin| (With a workaround for [this `winit` bug](https://github.com/emilk/egui/pull/8083#issuecomment-4206742668) applied.) <video src=https://github.com/user-attachments/assets/1e82583d-0c41-4f1c-98cf-0606bee5af05 />|——| | Wayland iBus Hangul |<video src=https://github.com/user-attachments/assets/8c9a0de1-9027-4b37-93a3-e9da0251d176 />|——| | Chromium (Chrome) macOS | Identical to `macOS builtin`. |——| | Safari macOS | Identical to `Windows builtin`. |——| | Firefox (Zen) macOS | Identical to `macOS builtin`. (ignoring the fact that the composition breaks when typing the second Hangul. (This bug predates this PR.)) |——| --------- Co-authored-by: lucasmerlin <hi@lucasmerlin.me>
260 lines
9.7 KiB
Rust
260 lines
9.7 KiB
Rust
//! The text agent is a hidden `<input>` element used to capture
|
|
//! IME and mobile keyboard input events.
|
|
|
|
use std::cell::Cell;
|
|
|
|
use wasm_bindgen::prelude::*;
|
|
use web_sys::{Document, Node};
|
|
|
|
use super::{AppRunner, WebRunner};
|
|
|
|
pub struct TextAgent {
|
|
input: web_sys::HtmlInputElement,
|
|
prev_ime_output: Cell<Option<egui::output::IMEOutput>>,
|
|
}
|
|
|
|
impl TextAgent {
|
|
/// Attach the agent to the document.
|
|
pub fn attach(runner_ref: &WebRunner, root: Node) -> Result<Self, JsValue> {
|
|
let document = web_sys::window().unwrap().document().unwrap();
|
|
|
|
// create an `<input>` element
|
|
let input = document
|
|
.create_element("input")?
|
|
.dyn_into::<web_sys::HtmlElement>()?;
|
|
input.set_autofocus(true)?;
|
|
let input = input.dyn_into::<web_sys::HtmlInputElement>()?;
|
|
input.set_type("text");
|
|
input.set_attribute("autocapitalize", "off")?;
|
|
|
|
// append it to `<body>` and hide it outside of the viewport
|
|
let style = input.style();
|
|
style.set_property("background-color", "transparent")?;
|
|
style.set_property("border", "none")?;
|
|
style.set_property("outline", "none")?;
|
|
style.set_property("width", "1px")?;
|
|
style.set_property("height", "1px")?;
|
|
style.set_property("caret-color", "transparent")?;
|
|
style.set_property("position", "absolute")?;
|
|
style.set_property("top", "0")?;
|
|
style.set_property("left", "0")?;
|
|
// Prevent auto-zoom on mobile browsers (requires at least 16px).
|
|
style.set_property("font-size", "16px")?;
|
|
|
|
if root.has_type::<Document>() {
|
|
// root object is a document, append to its body
|
|
root.dyn_into::<Document>()?
|
|
.body()
|
|
.unwrap()
|
|
.append_child(&input)?;
|
|
} else {
|
|
// append input into root directly
|
|
root.append_child(&input)?;
|
|
}
|
|
|
|
// attach event listeners
|
|
|
|
let on_input = {
|
|
let input = input.clone();
|
|
move |event: web_sys::InputEvent, runner: &mut AppRunner| {
|
|
let text = input.value();
|
|
// Workaround for an Android Gboard issue: after typing a word,
|
|
// the user has to delete invisible characters (whose count
|
|
// matches the length of the current suggestion) before actual
|
|
// characters are deleted, unless the focus has been reset.
|
|
//
|
|
// this issue appears to have been fixed in Gboard sometime
|
|
// between versions 14.7.09 and 17.0.12.
|
|
if !event.is_composing() {
|
|
input.blur().ok();
|
|
input.focus().ok();
|
|
}
|
|
|
|
if event.is_composing() {
|
|
// if `is_composing` is true, then user is using IME, for
|
|
// example: emoji, pinyin, kanji, hangul, etc. In that case,
|
|
// the browser emits both `input` and `compositionupdate`
|
|
// events.
|
|
// We handle the composition update here instead of in the
|
|
// `compositionupdate` event because the selection range
|
|
// has not yet been updated when `compositionupdate` fires.
|
|
|
|
let Some(text) = event.data() else { return };
|
|
let selection_start = input
|
|
.selection_start()
|
|
.unwrap_or(None)
|
|
.map(|pos| pos as usize);
|
|
let selection_end = input
|
|
.selection_end()
|
|
.unwrap_or(None)
|
|
.map(|pos| pos as usize);
|
|
let active_range_chars = if let Some(selection_start) = selection_start
|
|
&& let Some(selection_end) = selection_end
|
|
{
|
|
let text_utf16 = text.encode_utf16().collect::<Vec<u16>>();
|
|
let text_before_selection =
|
|
String::from_utf16_lossy(&text_utf16[..selection_start]);
|
|
let text_in_selection =
|
|
String::from_utf16_lossy(&text_utf16[selection_start..selection_end]);
|
|
let count_before_selection = text_before_selection.chars().count();
|
|
let count_in_selection = text_in_selection.chars().count();
|
|
Some(count_before_selection..count_before_selection + count_in_selection)
|
|
} else {
|
|
None
|
|
};
|
|
let event = egui::Event::Ime(egui::ImeEvent::Preedit {
|
|
text,
|
|
active_range_chars,
|
|
});
|
|
runner.input.raw.events.push(event);
|
|
} else {
|
|
if text.is_empty() {
|
|
return;
|
|
}
|
|
|
|
input.set_value("");
|
|
let event = egui::Event::Text(text);
|
|
runner.input.raw.events.push(event);
|
|
}
|
|
|
|
runner.needs_repaint.repaint_asap();
|
|
}
|
|
};
|
|
|
|
let on_composition_start = {
|
|
move |_: web_sys::CompositionEvent, runner: &mut AppRunner| {
|
|
// Repaint moves the text agent into place,
|
|
// see `move_to` in `AppRunner::handle_platform_output`.
|
|
runner.needs_repaint.repaint_asap();
|
|
}
|
|
};
|
|
|
|
let on_composition_end = {
|
|
let input = input.clone();
|
|
move |event: web_sys::CompositionEvent, runner: &mut AppRunner| {
|
|
let Some(text) = event.data() else { return };
|
|
input.set_value("");
|
|
let event = egui::Event::Ime(egui::ImeEvent::Commit(text));
|
|
runner.input.raw.events.push(event);
|
|
runner.needs_repaint.repaint_asap();
|
|
}
|
|
};
|
|
|
|
runner_ref.add_event_listener(&input, "input", on_input)?;
|
|
runner_ref.add_event_listener(&input, "compositionstart", on_composition_start)?;
|
|
runner_ref.add_event_listener(&input, "compositionend", on_composition_end)?;
|
|
|
|
// The canvas doesn't get keydown/keyup events when the text agent is focused,
|
|
// so we need to forward them to the runner:
|
|
runner_ref.add_event_listener(&input, "keydown", super::events::on_keydown)?;
|
|
runner_ref.add_event_listener(&input, "keyup", super::events::on_keyup)?;
|
|
|
|
Ok(Self {
|
|
input,
|
|
prev_ime_output: Default::default(),
|
|
})
|
|
}
|
|
|
|
pub fn move_to(
|
|
&self,
|
|
ime: Option<egui::output::IMEOutput>,
|
|
canvas: &web_sys::HtmlCanvasElement,
|
|
zoom_factor: f32,
|
|
) -> Result<(), JsValue> {
|
|
// Don't move the text agent unless the position actually changed:
|
|
if self.prev_ime_output.get() == ime {
|
|
return Ok(());
|
|
}
|
|
self.prev_ime_output.set(ime);
|
|
|
|
let Some(ime) = ime else { return Ok(()) };
|
|
|
|
if ime.should_interrupt_composition {
|
|
// no-op for now: currently, the text agent is sizeless, so any
|
|
// click shifts focus to the canvas, which naturally interrupts the
|
|
// composition.
|
|
}
|
|
|
|
let mut canvas_rect = super::canvas_content_rect(canvas);
|
|
// Fix for safari with virtual keyboard flapping position
|
|
if is_mobile_safari() {
|
|
canvas_rect.min.y = canvas.offset_top() as f32;
|
|
}
|
|
let cursor_rect = ime.cursor_rect.translate(canvas_rect.min.to_vec2());
|
|
|
|
let style = self.input.style();
|
|
let native_ppp = super::native_pixels_per_point();
|
|
|
|
// Clamp the input position within the canvas width to prevent unwanted horizontal scrolling.
|
|
let logical_canvas_width = canvas.width() as f32 / native_ppp;
|
|
let visible_x = cursor_rect.center().x * zoom_factor;
|
|
let clamped_x = visible_x.clamp(0.0, logical_canvas_width);
|
|
|
|
// Clamp the input position within the canvas height to prevent unwanted vertical scrolling.
|
|
let logical_canvas_height = canvas.height() as f32 / native_ppp;
|
|
let visible_y = cursor_rect.center().y * zoom_factor;
|
|
let clamped_y = visible_y.clamp(0.0, logical_canvas_height);
|
|
|
|
// This is where the IME input will point to:
|
|
style.set_property("left", &format!("{clamped_x}px"))?;
|
|
style.set_property("top", &format!("{clamped_y}px"))?;
|
|
|
|
Ok(())
|
|
}
|
|
|
|
pub fn set_focus(&self, on: bool) {
|
|
if on {
|
|
self.focus();
|
|
} else {
|
|
self.blur();
|
|
}
|
|
}
|
|
|
|
pub fn has_focus(&self) -> bool {
|
|
super::has_focus(&self.input)
|
|
}
|
|
|
|
pub fn focus(&self) {
|
|
if self.has_focus() {
|
|
return;
|
|
}
|
|
|
|
log::trace!("Focusing text agent");
|
|
|
|
if let Err(err) = self.input.focus() {
|
|
log::error!("failed to set focus: {}", super::string_from_js_value(&err));
|
|
}
|
|
}
|
|
|
|
pub fn blur(&self) {
|
|
if !self.has_focus() {
|
|
return;
|
|
}
|
|
|
|
log::trace!("Blurring text agent");
|
|
|
|
if let Err(err) = self.input.blur() {
|
|
log::error!("failed to set focus: {}", super::string_from_js_value(&err));
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Drop for TextAgent {
|
|
fn drop(&mut self) {
|
|
self.input.remove();
|
|
}
|
|
}
|
|
|
|
/// Returns `true` if the app is likely running on a mobile device on navigator Safari.
|
|
fn is_mobile_safari() -> bool {
|
|
(|| {
|
|
let user_agent = web_sys::window()?.navigator().user_agent().ok()?;
|
|
let is_ios = user_agent.contains("iPhone")
|
|
|| user_agent.contains("iPad")
|
|
|| user_agent.contains("iPod");
|
|
let is_safari = user_agent.contains("Safari");
|
|
Some(is_ios && is_safari)
|
|
})()
|
|
.unwrap_or(false)
|
|
}
|