Short: Next vi clone Author: Ali Gholami Rudi, Kyryl Melekhin Uploader: polluks+aminet sdf lonestar org (Stefan Haubenthal) Type: text/edit Architecture: ppc-morphos NEXTVI ====== Nextvi is a vi/ex editor. It can edit bidirectional UTF-8 text. NOTICE ====== As of 2022-01-01 Nextvi development has been driven to completion. No new features are being added, old ones polish and improve. Be sure to read Q1 in the FAQ section of this file before doing anything else. NAME ====== Nextvi tends to favor clean implementations over 100% adhering to POSIX like neatvi does. Thus the name "next", to signify that we are thinking outside the box. The notable changes that I am referring to (see below): 56, 60, 61. Some keybinds specified by POSIX were not implemented in original neatvi, nextvi might use them for some features below. NEATVI FEATURES --------------- - ex options: can be set using :se option or :se option=value can be unset using :se nooption td Current direction context. The following values are meaningful: * +2: always left-to-right. * +1: follow conf.c's dircontexts[]; left-to-right for others. * -1: follow conf.c's dircontexts[]; right-to-left for others. * -2: always right-to-left. shape If set (default), performs Arabic/Farsi letter shaping. order If set, reorder characters based on the rules defined in conf.c. hl If set (default), text will be highlighted based on syntax highlighting rules in conf.c. hll If set, highlight current line. ai As in vi(1). ic As in vi(1). - special marks: * the position of the previous change [ the first line of the previous change ] the last line of the previous change - special yank buffers: / the previous search keyword : the previous ex command - ex commands: :cm[!] [kmap] Without kmap, prints the current keymap name. When kmap is specified, sets the alternate keymap to kmap and, unless ! is given, switches to this keymap. :ft [filetype] Without filetype, prints the current file type. When filetype is specified, sets the file type of the current ex buffer. In nextvi :ft also reloads the highlight ft, which makes it possible to reset dynamic highlights created by options like "hlw". - new key mappings (normal): ^a searches for the word under the cursor. zL, zl, zr, and zR change the value of td option. ze and zf switch to the English and alternate keymap. gu, gU, and g~ switch character case. ^l updates terminal dimensions and redraws the screen. - new key mappings (insert): ^p inserts the contents of the default yank buffer. ^e and ^f switch to the English and alternate keymap. *vi(1) - for features unspecified refer to the respective page in the POSIX manual. NEXTVI FEATURES & CHANGES ------------------------- 1. Added unindent keybind: ^w ^w may also take vi_arg1 or motions as a region. 2. (Feature moved to patches) 3. Added linenumbers, keybind: # (relative numbers are placed after indent) Passing vi_arg1 to # enables permanent line numbers. 4. Added invisible character view, keybind: V 5. Added regex for changing spaces to tabs, keybind: vi 6. Added regex for removing r line endings and trail space/tab, keybind: vo 7. Changed behavior of ^a to change search direction when no more match. (see 49.) 8. Added fssearch, searches what is under the cursor or (last search kwd) in every file in the opened directory keybind: ^] or ^5 By default it checks every file, else filter is specified by :inc (see 32.) This works in compliance with other changes (see 49. 58.) If max number of available buffers reached, the last buffer will be reloaded; losing potential unsaved changes. Use fssearch for code navigation between files, jump to definitions / etc. fssearch runs 40% faster than busybox's grep, benchmark tested (time to find something that doesn't exist) on the linux kernel. 9. fssearch but going in reverse keybind: ^p 10. Added key to create a global mark for the current buffer, keybind: ^t There are 5 global marks. vi_arg1 == 0,2,4,6,8 creates a global mark. vi_arg1 == 1,3,5,7,9 switches to the mark. 0 (default) is a special mark created by fssearch used by ^p as a return point for if ^p can't find any more matches. 11. Added ex command "ea" which opens file using filename substring. Setting an extra parameter in form of a number will tell how many matches with similar filename to skip. For example file might be named "./path/bla/bla/file123.c" but you can open it just by "ea fi" if this is a uniquely matched filename. 12. Added ex command "fd" to set the directory for fssearch or "ea" ex command Running "fd" without an arg will recalculate directory info at current working dir. 13. Added numbered buffers to vi, default 10 and ex "b" command to show buffers and "b%d" to switch (where %d is the buffer number). Negative buffer numbers switch to temp buffers. Added new ex command "bx", where the argument will change the number of buffers allowed. If the number is lower than number of buffers currently in use they will be deallocated. Running bx without an arg will reset to default value. The default value is 10 or larger depending on the number of files specified in commandline arguments. Increasing the number of buffers may result in a positive effect on performance of fssearch (8. 9.) if the search keyword does not change. This is because vi will remember the position of previous match in the buffer avoiding redundant search. Improved buffer pathname expansion shortcuts. If you use character % or # in ex prompt they will substitute the buffer pathname. % substitutes current buffer and # last swapped buffer. Now it is possible to expand any arbitrary buffer by using % or # (no difference in this case) followed by the buffer number. Example: :!echo "%69" prints the pathname for buffer 69 (if it exists). % and # can be escaped normally if path expansion is not wanted. Added new ex command "bp" which allows the user to change path pointed by current buffer. For example :bp vi.c|e! will repurpose the buffer for vi.c 14. Added key to show buffers and switch buffer (to switch press corresponding 0-9 number) keybind: ^7 or ^_ If vi_arg1 is specified right before ^7 the buffer will be switched immediately which also happens to permit numbers > 9. 15. Added key to exit vi (not saving changes) keybind: qq or zz 16. Added key to goto first line keybind: gg 17. Added key to delete everything inside (cursor outside) "" keybind: di" or dc" 18. Added key to delete everything inside () keybind: di) or dc) 19. Added key to delete everything inside (cursor outside) () keybind: di( or dc( 20. Added key to change . to -> on cursor line keybind: vh 21. Added key to change -> to . on cursor line keybind: vg 22. Added new ex option "ish", this makes every "!" pipe command run though interactive shell so that stuff like aliases work. By default it is enabled, can be disabled if you need to for some odd reason via :se noish Every ex command can use # % substitution (see 13). Added a special substitution character ! which runs a pipe command. If the closing ! is not specified, the end of the line becomes a terminator. This makes any ex command be able to receive data from the outside world. To send data use :w !less notice how ! is escaped. Note: shell integration is one of the most powerful features of nextvi. 23. Changed the colors to be based on standard ANSI 16 colors desc in conf.c Colors can be customized up to 256 colors if the terminal supports it. 24. Added new syntax highlighting for C, js, html, css, diff... 25. Added key that splits the line (opposite of J) keybind: K K may also take vi_arg1, in which case it will repeat the operation on next line. 26. Added key that line wraps text to 80 line limit keybind: gq keybind: gw does the same but floors to the last word and is also much slower. gw is a macro, so set td=2 if there is bidi text, it makes a difference. 27. Added key that does multiline repeated edits keybind: v. This is based on the last commands and insertions and requires vi_arg1 for how many lines to repeat said operation. When last operation was 'i' or other commands that enter insert mode and some text, that text will be placed at the same offset on N number of lines specified by the vi_arg1. 28. Added ability to view the numbers for arguments that keys e,w,E,W,b,B may take. keybind: ^v Pressing again will change the key mode, specifying any vi_arg1 will exit the mode. This is a major step up to how navigation works in vi, it makes it so much easier to use because now you can see where you are going. These special numbers and their colors can be customized through conf.c. As of the latest git version, the feature will work correctly even if there is bidi text, double width characters and text reordering. 29. Added ability to change highlight dynamically. (via syn_reloadft();) 30. New ex option "hlw" which highlights every instance of word on the screen based on cursor position. Useful for when studying source code. 31. Added autocomplete in insert mode. Press ^g to index the current opened file. Then you can press ^n to cycle though the options, results are based on the contents of the file and the closest match to what you typed. Use ^r to cycle in reverse. By default, it will use big regex like [^;...]* to sort out all the punctuation chars from words and build a database of words. But in order to take full advantage out of the completion system, you can change this regex at runtime using new ex command "ac". For example say we don't want word completion and we want entire line completion instead, run :ac .* If the regex rule allows inclusion of nonalphanumeric or punctuation characters you won't be able to retrieve the string like it works with words by default. Automatically determining the position from which completion starts inside insert sbuf is ambiguous. More fine tuned control is needed and can be achieved using new keybind ^z in insert. Use ^z before you type out search term, this will set start position for completion, such that the options can be looped over inplace. ^z is a toggle, to disable it (without exiting insert) press twice in same place. When ^z is set ^u keybind will delete everything until ^z mark first, otherwise ^u operates normally (deletes everything). Autocomplete db is persistent throughout all buffers and it also has data duplication and redundancy checking such that same files can be indexed many times. Like ^g used to index file in insert mode, ^y can be used from insert to clear out the completion db. Running ex command "ac" with no argument will reset back to the default word filtering regex. You can find it's string in led.c as a reference. Using ^b from insert mode will display all possible autocomplete options. Added new ex option "pac". When enabled the autocomplete options will be automatically displayed. 32. Added ex command "inc" which sets the path filter using regex. Example 1: We want to get only files in submodule directory that end with .c extension: :inc submodule.*.c$ Example 2: Exclude the .git and submodule folders. :inc (^[!.git!submodule]+[^/]+$)|() Running "inc" without an arg will disable all filters. 33. Added file manager temp buffer keybind: - The buffer can be edited with any directory listing, for example via :1,1!find . or filled using built-in methods such as :fd. see (32. 12.) - New ex command "cd" changes working directory according to 1st arg. - Added key that opens the file based on text from the cursor. keybind: ^i or TAB 34. Added key to save current file keybind: ^k 35. The new special character "/" for ft in conf.c now signifies that the regex will be applied on any file. "/" is forbidden filename character on unix, the filetype that includes "/" is for internal use. 36. Added a window size signal handler for vi to redraw the screen automatically. 37. Added history buffer for ex commands when in vi prompt. keybind: ^b or vb (from normal) will open the buffer with all previous commands move the cursor to wanted command and exit buffer with qq will copy the command into prompt. To immediately execute the command exit with zz. You can also use this when half-way through some command and need to access normal mode to edit the command more efficiently. The filename is named "/hist/" so that it can't be written to file with ^k (by accident). To save history to file use :w yourfilename 38. Added key to grab the current word(s) under the cursor into prompt like so :%s/.../ keybind: vr (see also 49.) 39. Added ex option to change number of spaces in a tab (t) default is set to 8. use :se tbs=N (N is number of spaces) 40. Added partial support for multiline block regex, for example C multiline comments syntax highlight. 41. Made the lowest row to not waste any space when there are no messages and actually display the row. Passing vi_arg1 to ^g enables permanent status row. 42. Search via '/' or '?' automatically centers and redraws screen. This partly because change in 41 makes the bottom row behave dynamically and you might get search result on displayed on that row, which will be covered by search message instead. Also, centering is nice because you always know where to expect the result to be with your eyes. 43. Added terminal clean up on exit 44. Added a key to perform relative word replacements keybind: vt Specify vi_arg1 and the word(s) under the cursor will be placed into prompt, for example :.,.+5s// where 5 is vi_arg1. (see also 49.) 45. Improved single line performance by roughly 3x. Syntax highlight will not render anything beyond the terminal columns. If there is a line in the file that has say 200K characters, the performance will not degrade (except the order option set, which is by default). Please do not underestimate the difficulty of such endeavor, nextvi is required to take all these operations into account: 1. Bidi text direction. 2. Multibyte UTF-8. Double width chars. 3. Variable width tabulations that can change throughout the line based on surroundings. 4. Reordering of characters. (regex rules) 5. Syntax highlight. No other text editor has ever done all these requirements at once, therefore come to appreciate what is provided here and the level of performance. See also PERFORMANCE section at the end of the file. Because syntax highlight is bounded to the terminal dimensions some patterns may not be possible to match without rendering past the screen. To address this problem you may need some extra patch, see PATCHES section below. 46. Added key to abandon insert discarding any changes keybind: ^x That means there are 4 ways to exit insert|prompt now: ^x, ^c, ESC, kj RATIONALE: unlike others, ^x will not leave file marked as modified. 47. Added Russian keymap, and changed how xkmap_alt works, now z + vi_arg1 in normal mode will switch what keymap ^f key changes, so for example 1 = fa 2 = ru. If you need some other language kmap just make one yourself. Look into kmap.h it's really simple. kmap also has digraph support, that list has been expanded to include non printing characters. Naming resembles ASCII caret notation, so for example 03 is key ^c to be able to type 03 enter insert, use ^k to go into digraph mode, type then " c" (space+c) Essentially space is representing typeable ctrl key. The reason this is useful is if you are using @ key to make macros there has to be a way to type these control characters. Alternatively, it's possible to type the character literal using ^a + ^(insertkeyhere). In neatvi this was binded to ^v, currently ^v is used for 48. Nextvi shall keep both, the digraph way and ^a for clarity and better user reference, no more searching ascii keycodes online if kmap has it in the clear. 48. Improvement to change 37. Now when in prompt/insert ^v will bring up the latest command from history. Also the history works for searches via / or ? the same way. keybind vv does the same but from normal mode, to save time. Pressing ^v again goes to next string. 49. Added ability to get more than 1 word for keybinds ^a ^] ^p vr vt v/ specified by number (vi_arg1). Regex control chars will be escaped. vr vt v/ ^] ^p will grab word(s) only if vi_arg1 >= 1 otherwise the keybind will perform a default cursor independent action. 50. Added ability to edit the line while in insert mode such that backspace can delete all the characters on the line, when no more characters left the line will be wrapped onto the next one. This is behavior you can expect from 95% of editors, now nextvi is not an exception. The similar change was done for ^w keybind. 51. Added key to show the registers and their contents. keybind: R If the length of a register is bigger than the terminal, specify vi_arg1 to to shift the printing position by half the screen size vi_arg1 times. 52. Made feature of reverse text highlight toggleable via new ex option "hlr" 53. Added a key to disable autoindent. keybind: va This is necessary sometimes if you want to paste from system clipboard. 54. Removed full names of ex commands and options, (seriously, who uses that?) now only short and fast to type abbreviations work. 55. Substitute undo-redo point return to where command was issued initially if the affected area is greater than terminal rows. 56. Modified regex engine to support static lookahead expressions. For example [!abc] and [=abc] where ! is negated version of =. This will treat "abc" as (a && b && c) logically. It is possible to have multiple in one bracket expression as well. For example [!abc!cda!qwe] where each string delimited by the ! acts like a typical or operation i.e. [acq] with only difference of testing the extra characters ahead. To combine both standard bracket expression and lookahead in one, use ^ or ^= where ^ is negated and ^= is default. For example: [!abc^=123] characters after ^= match exactly how [123] would. 57. Added ^l key in insert to clean the terminal and start at first line, remove distractions and focus on typing. Useful when running ex via vi -e 58. Added v/ key in normal, which can grab the current word(s) under the cursor into prompt and set the current search string. If valid, the input will be used for all search related operations in vi. (see 49.) 59. Added ability to remember scroll amount for ^e and ^y keys (specified by vi_arg1). Advantage of ^e and ^y over using ^u and ^d is keeping the same vi_col position. 60. Removed bracket classes from regex. Not useful, hard to customize, buggy, error prone mess. Doesn't add any new functionality to the regex engine that can't be achieved without it. 61. Nextvi special character escapes work mostly the same way everywhere except the following situations: - Escapes in regex bracket expressions. This isn't posix but it solves couple of issues that were bugged previously, like escaping | in ex substitution command, properly counting number of groups in rset. - # % and ! characters have to be escaped if they are part of an ex command - A single back slash requires 2 back slashes, and so on. - rset_make() requires for ( to be escaped if used inside [] brackets. - In ex prompt the only separator is "|" character. It can be escaped normally but will require extra back slash if passed into a regular expression. 62. Added syntax highlighting continuation options. See the struct highlight in vi.h. The ^ anchor in the regular expression has an important property of being able to efficiently exclude some sub expression from being recomputed during the continuation. Take advantage of it when you can. 63. New ex option "hlp", will highlight the closest pair of symbols {([ from the cursor, the same way % key works. This feature exists to demonstrate complex syntax highlighting capabilities. 64. Ex options "hll", "hlw", "hlp" fully customizable in conf.c on per ft basis the same way you customize per ft highlighting. They must have highlight->func struct member set. If ft does not provide a spot in hl, the latter feature will not work on that ft, regardless of ex options being set. 65. Added a key to quickly access :! prompt. keybind: v; Removed "make" ex command. Commands like these are not wanted, nextvi shall provide a more general purpose solution for the user, like the keybind v; for example. 66. Added new ex option "grp". The following allows definition of target search group for / ? n N (31.) autocomplete and ex substitution. This becomes necessary when the result of regex search is to be based on some group rather than default match group. For example you want to search for the whole line but exclude the tabs at the beginning of the line, use regex like this: [ ]+(.[^ ]+) since only the capture result for 2nd group matters use the "grp" like this: :se grp=4 .The number 4 is important, it is calculated using: grpnum * 2, to get the second group number do: 2 * 2 = 4. The first group is always 2, 1 * 2 = 2, :se grp=2 gets you back to default group/behavior. 67. Undo and Redo commands (u,^r) may take optional vi_arg1 which repeats the operation N times. 68. Search motions do not terminate with error if the count is greater than number of instances found. Last possible match will be used. Important when you don't know exactly how many matches there are, does not mean there aren't any at all, greedy behavior opens up new use cases. 69. New ex command "tp", when arg given immediately executes the macro defined by arg. It can run any vi normal command and execute insert statements. The advantage of tp over traditional macros is in the ability to bypass the macro queue and run independently. In a way, macro executed by tp exercises the same causality as running C code directly. 70. Added key to list through the buffers. keybind: ^n vi_arg1 changes the direction of ^n. 71. In insert mode keybind ^o allows access to a secondary ex prompt or new vi instance so that it becomes possible to switch to vi from ex or postpone some operation, do something else and come back later. This feature is implemented using functionality provided by 69. and exists to prove and demonstrate that vi is to some extent REENTRANT as in single threaded recursion case, where operations can be safely executed in a separate instance. Still, a great amount of care and understanding is needed when making such complicated macros. 72. Added keybinds ^ and ^] to control the default register for pasting (^p) in insert mode. ^ resets to default yank register. ^] changes register to the next available, between 0 and 9. Contents of the register are displayed on the bottom row. In nextvi, register 0 stores previous value of default register if operation is atomic and did not include a whole line, else register 1 takes its place. LESSER KNOWN FEATURES --------------------- - "Ever tried reading the source code?" Yes, that is a lesser known feature, what did you expect? Jokes aside (with a level of truth to it), these features exist in many other vi implementations but neither man pages cover their functionality in an understandable language, describe it here instead. - @@ macros: 1. Type out the macro or load from file such that it is in some vi buffer. 2. Use keybind "ayy on the macro, this will store it in register 'a' 3. Use @a to play it back, where a stands for that 'a' register 4. @@ repeats the last macro on next line - substitution backreference: This inserts the text of matched group specified by x where x is group number. Example: this is an example text for subs and has int or void :%s/(int)|(void)/preafter this is an example text for subs and has preintafter or void :%s/(int)|(void)/pre2after/g this is an example text for subs and has prepreafterafter or prevoidafter - ex global command: same syntax as ex substitution command, but instead of replacement string it takes an ex command after the / / enclosed regex. Example: remove empty lines :g/^$/d Try doing similar with substitution command - will not work as removing 'n' without deleting the line is invalid, but it will work with global command. - search motions: ? and / searches have the ability to be used as motions. This seems very counter intuitive and one would have never ever figure out that this feature even exists, unless noted. Even if you read the source code it's very easy to miss. How to use: optionally specify vi_arg1, specify the motion using it's keybind, then do / or ? and type out the search term. The motion ends on the first match by default (no vi_arg1 specified). The optional vi_arg1 determines how many matches of the term to skip until the motion ends. Example: you see that the next 10 lines have the word "int" which is included 3 times. You want to delete text until the 3rd instance of "int" keybind would be 3d/int . Likewise you can opt out of the "specify motion" part and just use / or ? with vi_arg1 to perform specific searches. - Majestic EXINIT environment variable At the zenith of your vi/nextvi education you'll find that EXINIT can be used to achieve arbitrary level of customization. Using new ex command "tp" any sequence of vi/ex commands can be performed at startup. This is where real "groking vi" starts. Example 1: There is a dictionary file (assume vi.c), which we always want to have indexed at startup for autocomplete feature in 31. export EXINIT="e ./vi.c|tp i|u|bx 1|bx" The last "bx" commands delete the vi.c buffer. "u" can be used to unmark vi.c as being modified. To keep it around as a buffer remove the "bx" commands. Example 2: Load your shell's history into vi's history buffer and adjust the data such that it is usable by appending ! at the beginning of command and escaping the "|" pipes the way ex prompt expects them (see 61.) export EXINIT="e /root/.ash_history|tp yG:p:%s/^/!\|%s/ | / \\\| /g qq|bx 1|bx|ft" Congratulations, vi has unofficially replaced your shell's frontend. Example 3: Setup some custom @@ macros in your favorite registers. export EXINIT="e|tp io{ }kA|tp 1G|tp 2"ayy" This macro gets loaded into register a, when @a is executed the macro will create { and closing } below the cursor leaving cursor in insert mode in between the braces. This is something you would commonly do in C like programming language. - CRITICAL - the new line inside the EXINIT string is literal, it's best to store the export command in a .sh file and set it using: $ . ./init.sh Otherwise this examples 2 and 3 may not work!!! To suppress EXINIT variable use -s commandline flag. - Uppercase registers In vi uppercase registers append to the lowercase register instead of overwriting the register completely. This is very useful, for example, use global and yank ex commands together: :g/searchterm/ya A Now we can use "ap and paste all the lines matched by the regex. Depending on the usecase simply :g/searchterm/ya or :g/searchterm/p may also be enough. PATCHES ------- New functionality can be obtained through optional patches provided in the patches branch. If you have a meaningful contribution and would love to be made public the patch can be submitted via email or github pull request. FAQ: ---- Q1: What's the best way to learn vi/nextvi? A1: First ensure you know basic movements hjkl this would suffice. Start reading vi.c don't worry about the rest of this readme until later. Running ./vi vi.c use / and n N keybinds for search and look for switch cases. The keybinds are encoded to be intuitive. Once you find some case that looks like a keybind read the code, if you don't understand try to reproduce the keybind to better understand what the code does. You have to do this, if you omit this step you will never be able to realize the full potential that the software provides. It's not desirable to live in the dark using this software for the next 10 years only to find that for example, ^p in insert mode exists and is very useful. If you can't figure how to use the keybind at least you would know at the back of your mind that there is something there, realization will come later. It's better to skim look through the switch cases than to never even open it. This isn't an excuse, but a deliberate design goal, where the user reads the code in order to achieve the full control he/she desires. LOC: +--------------+---------------------+ | 569 kmap.h | keymap translation | | 428 vi.h | definitions/aux | +--------------+---------------------+ | 577 uc.c | UTF-8 support | | 318 term.c | low level IO | | 298 conf.c | hl/ft/td config | | 652 regex.c | extended RE | | 590 lbuf.c | file/line buffer | | 1083 ex.c | ex options/commands | | 2085 vi.c | normal mode/general | | 685 led.c | insert mode/output | | 384 ren.c | positioning/syntax | | 6670 total | wc -l *.c | +--------------+---------------------+ The code is devised to be unquestionable. You will be able to read, understand and modify this code faster. Come back to this readme regularly as it documents more advanced behavior. Q2: What does it mean when I call feature X a macro? A2: It's the kind of shortcut that does not change the core functionality, but rather reuses the core functionality. Usually macro features are implemented in 1 or a few lines of code. Notably, they tend to use function term_push(), but it's not required. Because they are macros they may run suboptimally or not handle every possible edge case. When calling a macro feature from another macro, the results are pushed back, which means the macro feature will always execute last, see (69.). These features are considered a macro: 5. 6. 16. 17. 18. 19. 20. 21. 26. 27. 34. 46. 50. 65. 71. Q3: Keybind with CTRL does not work? A3: vi is reading ASCII codes sent by the terminal. Depending on the keyboard, the ASCII code could be another key combination. It was reported that "^^" (Ctrl + ^) can be achieved on some system with "^6". If something doesn't work, have a look at the layout of an american/british keyboard and try to reproduce the keybind as if you have an american/british keyboard. Q4: Why nextvi instead of vim? A4: I prefer customization in source code, Vim is considered harmful. Q5: Why not distribute as patches, like on suckless.org? A5: It's hard to maintain. Simply put, there are too many changes to keep track of if compared to original neatvi. Q6: Why are keybinds encoded as pure switch cases instead of more suckless.org style keybind function table dispatch? A6: Because we want small efficient code that is easy to write. In nextvi many keybinds interoperate so that they can do multiple tasks at various conditions. Use of goto is encouraged, it is simply impossible to achieve this behavior in a sensible way otherwise. Suckless code style philosophy crumples when requirements are as complex as what vi needs to be able to do. In other words, it depends - but if the standard of C provides means to implementing things cleaner and faster you should use the smallest form factor possible. In retrospect, it may be harder to find the implementation itself, but once you do there is nothing else hidden from you. The result is unabstracted program control flow that can be easily read and modified reducing the risks of unforeseen side effects. Q7: General philosophy? A7: User is programmer, hacker culture. In most text editors, flexibility is a minor or irrelevant design goal. Nextvi is designed to be flexible where the editor adapts to the user needs. This flexibility is achieved by heavily chaining basic commands and allowing them to create new ones with completely different functionality. Command reuse keeps the editor small without infringing on your freedom to quickly get a good grasp on the code. If you want to customize anything, you should be able to do it using the only core commands or a mix with some specific C code for more difficult tasks. Simple and flexible design allows for straight forward solutions to any problem long term and filters bad inconsistent ideas. Q8: Something, something - pikevm A8: Pikevm is a complete rewrite of nextvi's regex engine for the purposes of getting rid of backtracking and severe performance and memory constraints. Pikevm guarantees that all regular expressions are computed in constant space and O(n+k) time where n is size of the string and k is some constant for the complexity of the regex ie. number of state transitions. It is important to understand that it does not mean that we run at O(n) linear speed, but rather the amount of processing time & memory usage is distributed evenly and linearly throughout the string, the k constant plays a big role. If you are familiar with radix sort algorithms this follows the same idea. Q: What are the other benefits? A: For example, now it is possible to compute a C comment /* n */ where n can be an infinite number of characters. Of course this extends to every other valid regular expression. Q: New features pikevm supports? A: Additionally, pikevm supports PCRE style non capture group (?:) and lazy quantifiers like .*? and .+?? because they were easy to implement and allow for further regex profiling/optimization. Q: NFA vs DFA (identify) A: pikevm = NFA backtrack = DFA Q: What's wrong with original implementation? A: Nothing except it being slow and limited. My improved version of Ali's DFA implementation ran 3.5X faster in any case, however I found a bug with it where quested "?" nested groups compute wrong submatch results. To fix this problem, it would require to undo a lot of optimization work already done, basically going back to how slow Ali's implementation would be. The reason this was spotted so late was because this kind of regex wasn't used before, so I never tested it. Other than that I think submatch extraction is correct on other cases. Pikevm does not have this bug, so it will be used as main regex engine from now on, unless dfa ever finds a proper fix. Honestly, this change isn't so surprising, as I was working on pikevm a few months prior, to favor a superior algorithm. You can still find that code here (likely with no updates): https://github.com/kyx0r/nextvi/tree/dfa_dead As a downside, NFA simulation looses the DFA property of being able to quickly short circuit a match, as everything runs linearly and at constant speed, incurring match time overhead. Well optimized DFA engine can outperform pikevm, but that is rather rare as they got problems of their own. For example as independently benchmarked, dfa_dead runs only 13% faster than pikevm and that is stretching the limit of what is physically possible on a table based matcher. Can't cheat mother nature, and if you dare to try she's unforgiving at best. Supplementary reading by Russ Cox: https://swtch.com/~rsc/regexp/regexp1.html PERFORMANCE ----------- Stress test: 1. Compile both versions with gcc -O2 -g (no customizations done) 2. Open Nextvi's vi.c and hold ^d until the end of the file 3. Capture the results with cachegrind 4. To find out what these values mean see: https://valgrind.org/docs/manual/cg-manual.html NEATVI: -------------------------------------------------------------------------------- | I refs: 1,030,137,070 | I1 misses: 122,660 | LLi misses: 1,298 | I1 miss rate: 0.01% | LLi miss rate: 0.00% | | D refs: 672,331,702 (373,790,802 rd + 298,540,900 wr) | D1 misses: 473,511 ( 215,088 rd + 258,423 wr) | LLd misses: 7,836 ( 1,116 rd + 6,720 wr) | D1 miss rate: 0.1% ( 0.1% + 0.1% ) | LLd miss rate: 0.0% ( 0.0% + 0.0% ) | | LL refs: 596,171 ( 337,748 rd + 258,423 wr) | LL misses: 9,134 ( 2,414 rd + 6,720 wr) | LL miss rate: 0.0% ( 0.0% + 0.0% ) -------------------------------------------------------------------------------- NEXTVI: -------------------------------------------------------------------------------- | I refs: 364,197,180 | I1 misses: 93,186 | LLi misses: 1,228 | I1 miss rate: 0.03% | LLi miss rate: 0.00% | | D refs: 142,125,211 (100,909,090 rd + 41,216,121 wr) | D1 misses: 140,991 ( 81,656 rd + 59,335 wr) | LLd misses: 6,043 ( 1,855 rd + 4,188 wr) | D1 miss rate: 0.1% ( 0.1% + 0.1% ) | LLd miss rate: 0.0% ( 0.0% + 0.0% ) | | LL refs: 234,177 ( 174,842 rd + 59,335 wr) | LL misses: 7,271 ( 3,083 rd + 4,188 wr) | LL miss rate: 0.0% ( 0.0% + 0.0% ) -------------------------------------------------------------------------------- Further optimization: 1. To create the most optimal exe, enable PGO optimizations by compiling via ./build.sh pgobuild which can lead to a significant performance boost on some application specific tasks. Feel free to adjust build.sh and the sample data on which it's being trained on, though default probably already good enough. 2. To improve nextvi's performance, shaping, character reordering, and syntax highlighting can be disabled by defining the EXINIT environment variable as "se noshape | se noorder | se nohl | se td=+2". Favorite quotes: -------------------------------------------------------------------------------- "All software sucks, but some do more than others." - Kyryl Melekhin "Educated decisions assert the quantitative quality first." - Kyryl Melekhin "It’s possible that I understand better what’s going on, or it’s equally possible that I just think I do." - Russ Cox "Vigorous writing is concise. A sentence should contain no unnecessary words [and] a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all his sentences short, or that he avoid all detail and treat his subjects only in outline, but that every word tell." - Elements of Style, William Strunk, Jr. - 1918 -------------------------------------------------------------------------------- CREDITS ============= Programming ------------- Kyryl Melekhin (kyx0r) Ali Gholami Rudi (aligrudi) Documentation / Design / Testing ------------- Kyryl Melekhin (kyx0r) Proofreading ------------- Kyryl Melekhin (kyx0r) Cédric (Vouivre) Special Thanks ------------- Ali Gholami Rudi (vi https://github.com/aligrudi/neatvi) ArmaanB (bsd test) aabacchus (build.sh) illiliti (build.sh) git-bruh (feedback) and all users, posters & haters :/