Skip to content

Instantly share code, notes, and snippets.

@XVilka
Last active August 9, 2024 06:36
Show Gist options
  • Save XVilka/a0e49e1c65370ba11c17 to your computer and use it in GitHub Desktop.
Save XVilka/a0e49e1c65370ba11c17 to your computer and use it in GitHub Desktop.
BiDirectional Text

This gist will show the support of BiDirectional text in the terminal emulators and console programs. You can read more about the standardization efforts at the dedicated page of FreeDesktop Terminal BiDi working group.

How to test

Logical Order ◀ ◀ ◀ RTL LTR ▶ ▶ ▶
WHAT IS UNICODE؟ in arabic in arabic ؟EDOCINU SI TAHW ؟EDOCINU SI TAHW in arabic
ما هو الترميز الموحد يونيكود؟ in Arabic ما هو الترميز الموحد يونيكود؟ in Arabic

Supporting BiDirectional text

According to a new standard

Please note, that specification is preliminary yet, and VTE implementation is still work in progress:

Incompatible and incomplete implementations

Supporting BiDirectional text programs, but incompatible

Not supporting BiDirectional text

Terminal multiplexers

Not supporting BiDirectional text programs

@egmontkob
Copy link

I've spent the last few months studying and evaluating the only document about BiDi in terminals that I could locate, as well as existing implementations. Unfortunately I have severe problems with all of them.

The most important problem with all the self-claimed BiDi-aware terminals is that they always apply the Unicode BiDi algorithm on the contents, giving no way for applications to switch it off. This way they literally, mathematically provably become a platform on top of which it's impossible to implement proper BiDi-aware text viewing/editing experience. (In most terminals you can disable BiDi in the settings, but it's an unacceptable user experience if people have to do so every time they start up or quit a text editor.)

They also work on lines of text rather than paragraphs, lack support for overall RTL direction, and suffer from various one-off problems as well.

Thus I'd make the bold claim that the terminals mentioned above as supporting BiDi actually don't. Supporting BiDi doesn't mean unconditionally doing it, since it does about as much harm as good. Supporting BiDi means (among plenty of other things) knowing when to do it and when not to.

I came up with a draft specification on how BiDi should be done in terminal emulators, it's at https://terminal-wg.pages.freedesktop.org/bidi/ . Feedback about this spec is welcome over there. I'm making a somewhat bold, but well supported claim that only terminals that implement BiDi according to this spec should be called BiDi-aware.

Recently Mintty did a couple of improvements, partially along the lines of the new specs. I haven't yet had a chance to evaluate what it supports now and what it still doesn't.

@DirectorX
Copy link

Rxvt-unicode with fribidi (Perl) plugin supporting bidirectional text

@XVilka
Copy link
Author

XVilka commented Jul 2, 2019

Just an update - was recently implemented in libvte by @egmontkob: https://terminal-wg.pages.freedesktop.org/bidi/implementations.html#vte

@faridcher
Copy link

https://github.com/aligrudi/neatvi a fork of VI works great and may be included in the list of programs.

@zzgraph
Copy link

zzgraph commented Apr 4, 2020

I suppose Termite does not support bidirectional text rendering because their code is not compatible with vanilla libvte.

@nemya9066
Copy link

Please Remove Termite from the list it doesn't support bidirectional text they use a different vte library.
Tested on arch linux.

@XVilka
Copy link
Author

XVilka commented Sep 8, 2020

Removed Termite, thanks!

@foodornt
Copy link

How may someone contribute to one of these terminals to add BiDi support? if anybody can reference a starting point, may it be how to shape fonts, how to implement BiDi, or even the part of code of these libvte based terminals that makes it able to view BiDi text, that would be appreciated.

@SafwanLjd
Copy link

Vim actually supports bidirectional text, all you have to do is use a terminal that supports it (or BiCon) and put :set arabicshape! in your .vimrc

@progandy
Copy link

WezTerm has premliminary BiDi support in recent releases: wez/wezterm#784 (comment)

@avidseeker
Copy link

Compiled programs support for Bidi in this ArchWiki article.

@ctrlcctrlv
Copy link

ctrlcctrlv commented Jun 30, 2023

There's finally some movement on this; Unicode Technical Committee №175 approved a Working Group to be called the Terminal Complex Script Support Working Group (TCSS WG) after Microsoft asked for it in L2/23-107.¹

Microsoft wants to do it in a libvte-incompatible way so I plan to attend every meeting of the WG I can just to discuss…why, it seems they may not have even known about prior art. My document is L2/23-153.²

Footnotes

  • ¹ L2/23-107
    Li, Renzhi et al. Proper Complex Script Support in Text Terminals. 2023-04-24.
  • ² L2/23-153
    Brennan, Fredrick R. Opposition to and Comment on L2/23-107. 2023-06-30.

@ninjalj
Copy link

ninjalj commented Jul 2, 2023

@ctrlcctrlv : Re: "Opposition to and Comment on L2/23-107", the BiDi in Terminal Emulators draft proposal is by Egmont Koblinger.

@ctrlcctrlv
Copy link

Yes, sorry, my mistake; @XVilka told me this already via email; but the Unicode office is closed on weekends :-) I'll submit a revised (“R”) version to L2 via docsubmit@ next week.

@hemedani
Copy link

hemedani commented Jul 2, 2023

what about wezterm ?

@ctrlcctrlv
Copy link

What about it?

This output is wrong:

image
For «Wikipedia», ויקיפדיה is the correct writing order, not הידפיקיו.

It fails the two major tests of the debu.tanuki.agency test:

image

It accepts the fribidi'd Hebrew as-is meaning it's on par with kitty (i.e. broken) and mangles the Devanagari worse than Konsole or GNOME Terminal but slightly better than Kitty yet far behind mlterm.

@wez
Copy link

wez commented Jul 9, 2023

@ctrlcctrlv could you share with me how to run the tests you ran with wezterm?

@ctrlcctrlv
Copy link

@wez Sure!

Do:

ssh [email protected]

No password. That's not how I ssh to デブ.狸.agency and is there as a joke against would-be crackers and skiddies, but it's useful for this too! (As that message is just my MOTD.)

@anonimo0-0
Copy link

So is there a terminal that supports bidi yet? Gnome terminal isn't that good. I've tried to run Neovim inside it, and got the wrong behavior. mlterm seems to be the one with the best behavior out of all the ones that I've tried, but its documentation is lacking and it doesn't look very nice out of the box.

@ctrlcctrlv
Copy link

that is the state of the art. mlterm does the best job.

@cben
Copy link

cben commented Aug 21, 2023 via email

@ctrlcctrlv
Copy link

perhaps bidi better on win32 but i see dotted circles

fails both deva and hebr

image

@mmstick
Copy link

mmstick commented Dec 20, 2023

cosmic-term supports bidirectional text and ligatures via cosmic-text: https://fosstodon.org/@soller/111602927867586005

@wolandark
Copy link

wolandark commented Jul 24, 2024

Konsole is actually doing a great job at bidi, in fact it is the only terminal that renders the RTL text from the right edge of the terminal window.

See this screenshot from Konsole displaying English and Persian bible text in one window.

I'm not sure if others can render LTR and RTL text in such fashion.

The settings window

@mintty
Copy link

mintty commented Jul 24, 2024

I'd say all terminals I've seen to do bidi do it that way - how else? To check, however, please upload your test file.

The konsole feature "Table characters BiDi mode override" might be interesting but unfortunately it is not really specified (which characters? examples?); maybe I'll submit them an issue.

@MoSal
Copy link

MoSal commented Jul 24, 2024

@wolandark

In any VTE-based terminal, echo '\e[?2501h' to enable auto-direction, and you should get the same behavior.

This comes from terminal-wg's BiDI draft proposal.

@wolandark
Copy link

@MoSal Neat! I didn't know that. After running the escape sequence, XFCE terminal also behaves the same as Konsole does.

@wolandark
Copy link

@mintty here is a sample of the text (though what I shared in the screenshot was being fetched from an API).

As mentioned above, doing as MoSal suggested, makes XFCE terminal (which is VTE) behave the same way as Konsole, so I guess my defense of Konsole is relevant, though in my defense there is no need to run any escape sequence in Konsole to get this behavior.

@MoSal
Copy link

MoSal commented Jul 24, 2024

though in my defense there is no need to run any escape sequence in Konsole to get this behavior.

FWIW, LTR instead of AUTO as a default paragraph direction was a design choice which I, as a native speaker of an RTL language, tend to agree with.

@mintty
Copy link

mintty commented Jul 26, 2024

The konsole feature "Table characters BiDi mode override" might be interesting

It appears that mintty already has a mode that seems to render table layout like konsole: echo -e "\e[?2501l".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment