For use of the LaTeX discussion group. The purpose of this list is to discuss the development of the successor to LaTeX version 2e. Please direct general questions about version 2.09 and 2e to other lists. It is, however, perfectly allowed to point out deficiencies which should be improved in the next version. Note: ----- If mail to a list member is rejected three times, the corresponding mail address is deleted without further notice.
I’ve been thinking: since Lua is already involved in the build process, by way of l3build, wouldn’t it be reasonable to use a lua script to preprocess Unicode data into forms that are easily consumed by LaTeX during the format-building process?
On 16/03/2020 17:01, Kelly Smith wrote: > Hello! > > I’ve been thinking: since Lua is already involved in the build process, > by way of l3build, wouldn’t it be reasonable to use a lua script > to preprocess Unicode data into forms that are easily consumed by LaTeX > during the format-building process? > > Warmly, > Kelly >
On 16/03/2020 20:27, Henri Menke wrote: >> Now, there is more data being loaded today than when I did that work, and >> some of it is in LuaTeX so could be done Lua-only. It's also possible that >> the Perl script was sub-optimal, or that as part of a general 'install' >> function the time would not really show. However, XeTeX needs the data, so >> one is still looking at having to explicitly pre-process in Lua. Moreover, >> most of the time taken for format-building is not about reading Unicode >> data. With LuaTeX, pre-loading expl3 does cut
>As format-building is all about saving time for 'normal' runs, I'm not >seeing there is a massive need to speed up the process. I know there is >one engine in development that doesn't use format files, so that might >be a place to consider things, but I think we'd need a strong case to >alter the approach for XeTeX/LuaTeX (pdfTeX, ...). > >Joseph
On 16/03/2020 22:54, Kelly Smith wrote: >> As format-building is all about saving time for 'normal' runs, I'm not >> seeing there is a massive need to speed up the process. I know there is >> one engine in development that doesn't use format files, so that might >> be a place to consider things, but I think we'd need a strong case to >> alter the approach for XeTeX/LuaTeX (pdfTeX, ...). >> >> Joseph > > Sorry, I should’ve clarified: the point of preprocessing the data wouldn’t > be to speed up anything, instead, the point would be to
On 16/03/20, 17:32, Joseph Wright wrote: > On 16/03/2020 17:01, Kelly Smith wrote: > > Hello! > > > > I’ve been thinking: since Lua is already involved in the build process, > > by way of l3build, wouldn’t it be reasonable to use a lua script > > to preprocess Unicode data into forms that are easily consumed by LaTeX > > during the format-building process? > > > > Warmly, > > Kelly > > > > It depends on the outcome you are after. > > The original loading method for Unicode data in XeTeX was via
> Similarly, a lot of work is done by Javier for babel using the CLDR, but > many of the outcomes are not amendable to scripting: it's about how you set > up the TeX 'just right'.
Before starting my work on the CLDR I was overly optimistic. Just a few lua scripts with a json or xml library seemed enough. But after my initial efforts it was clear a lot of thing had to done by hand. Currently I write ad hoc Python scripts (batteries included, you know) for specific tasks (for example, I few days ago I added
