readline: Update fq fork

[wader]

Rebase on master
Cherry-pick chzyer/readline#202

[wader]

@thushan Would be great if you could do some Windows testing of this change. Make sure colors work and mess around in the REPL.

[tpodowd]

Thanks for trying this out guys. My use case is not Windows but hopefully I didn't miss too much :-)

[thushan]

will get our QA folks to use this new build and test it out over the weekend and let you know Monday (AEST)

[wader]

@thushan Great!

@tpodowd When i was doing development on Windows i ended up with this dev setup https://github.com/wader/fq/blob/master/doc/dev.md#setup-docker-desktop-with-golang-windows-container which is very nice. If your on macOS you can have one shared source tree and then have three terminals using docker to build/run on macOS, linux and windows, made iterating a lot faster.
Sadly every time i've done it was initially quite a mess to find a windows version that matches a golang docker image build.

[wader]

If i remember correctly the terminal stuff on windows works quite well, but i guess there must be some terminal control codes translation going on? so in the end i guess it's best to try directly on a windows desktop.

[tpodowd]

@wader @thushan I presume there is no trouble with this so far? If you guys are happy, maybe you can stick another comment on my mail pull request to get it back on the radar again.

[wader]

@tpodowd i've used the changes on macOS and some Linux for some days and haven't noticed anything bad, which is probably a good sign as before i think there were some weirdness happening at times.

[thushan]

Apologies I thought that one of the guys will have updated this thread - but he's got COVID it seems, so he'll be out for another week.

Good news is that they haven't noticed any changes from the build I built from master which has your changes @tpodowd. They tested it on Windows 10 and Windows 10 with Windows Terminal (internal development on their machines).

The feedback was:

• existing workflow continues, so nothing broke - spaces etc remain.
• seems like the redraw is faster in master than previous builds (TF: Is this something that was expected? noticed on the Windows 10 + Windows Terminal build, but unsure if it's due to the new build or update to WT)

[wader]

@thushan thanks, any idea what features they usually use/try? i'm mostly concern about interactive REPL usage, editing, line wrap, resize, completion etc and maybe colors.

[tpodowd]

Hi @thushan - yeah, it should be faster. Previously, every time you typed a character, readline would clean up everything, then rewrite everything and then adjust the cursor using back spaces. New version, just appends characters and doesn't redraw unless it you are typing in the middle and uses 2 control sequences to jump to the correct cursor point. I don't expect anything to break regarding colour or spacing. One difference I noticed between windows code and linux code was that if you resize the window, the windows readline doesn't pick up the new screen size so its not going to work well. This is not a new issue though and was in the previous build. I just didn't fix it as my use case is linux.

[thushan]

@wader
They mostly use it to analyse video files and some image files. the build agents are picky about parsing the outputs of fq and it hasn't broken anything with this master release. the QA folks do use REPL to analyse things within the file during times we mess things up (which currently is quite regularly). Colours, wrapping, behaviours are the same. Unsure if they edit or use completion tho. Will email Monday.

@tpodowd
Ah brilliant, I did notice the resize issue (just now when I went to resize a Windows Terminal window running Powershell).

v0.0.7 (latest)

v0.0.7.x (master)

Same behaviour when windows resized thrice. Looks good to me.

[wader]

@thushan Thanks for checking. Sorry by edit i just ment line editing, backspace, move around text cursor and so on ... but if they use the REPL i guess so :)

[wader]

@thushan btw i gave a talk about fq some months ago, some history, implementation details and such but there is also a demo at the end how i use it to poke around and query things, also shows some cli and REPL tricks https://www.youtube.com/watch?v=GJOq_b0eb-s

[thushan]

also shows some cli and REPL tricks https://www.youtube.com/watch?v=GJOq_b0eb-s

Aha, I did pass that on to the team, i was one of the 7 that liked that video too :) Very detailed overview!

[wader]

Aha, I did pass that on to the team, i was one of the 7 that liked that video too :) Very detailed overview!

👍 😄 Thanks!

[tpodowd]

Sounds like the patch is working well for you guys on Windows. That's great. I've been testing it quite a lot too on linux and seems to be solid right now for me. I'm going to ping the main pull request again so see if it can be merged. Perhaps you guys can also add a comment.

[tpodowd]

Hi @wader - do you use tab completion in readline? I just added another PR which fixes some completion crashes and adds a Pager Mode for when there are too many completion candidates (similar to bash). Also added some more aggregate completion so that tab completes the common part of any candidates returned. Anyway - thought you might be interested. My other PR is not merged yet so the new PR includes the redraw patch also.

[wader]

Nice, yeap fq:s REPL uses readline completion quite extensively. Will take your PR for spinn later today or tomorrow

[tpodowd]

ok great. Sorry to ping you on this old thread. If you have any feedback, hit me up on the PR.

[wader]

Will do, and no worries, email, twitter DM or whatnot work fine for me also if want discuss something