Road Map for Future Development of PDF::Builder              11 September 2019

In order to encourage others to contribute code and/or algorithms to the
effort, I am publishing this road map of where I would like the product to go.
Please, no copyrighted code or patented algorithms, unless the owner releases
them under an Open Source license! The content of this road map is open to
discussion, too, either on the GitHub bugs list (feature requests with the
"enhancement" label or "general discussion" label) or under the forum (Feature 
Requests or General Discussions). If you have a one-off suggestion, there is
a contact link on the forum so you don't have to sign up for either the forum
or GitHub to be heard.

If there are no contributions on something, I reserve the right to write my
own modules (dependent on PDF::Builder) and sell them.

I make no promises that any of the following items will be implemented; it
depends on how much free time I can come up with, and how many people chip in
to help with code and algorithms. I'll be happy to discuss coding specific
requirements for money/donations (but the result is still free software). 

The assignment to section I or II is somewhat arbitrary, and an item could move 
from one to the other. Some of these items are already listed in bug reports, 
or as feature requests. There is no particular order to these items (i.e.,
they are not ranked by priority).

=============================================================================
I. Items to add to the core product

These are things that should be in the base PDF::Builder product, as everyone
will need them (or, it would be cleaner to have it in the base rather than as
an add-on separate module).

A. ## DONE ## release 3.018
   Proper TTF/OTF support (RT 113700), especially ligature replacement and
   complex script alphabets using GSUB and GPOS information. I've been looking
   at Pango and directly using HarfBuzz, but both look to be a lot of work.

   I know of a developer tinkering with an add-on layer using Pango for both 
   PDF::API2 and Builder, but it's not clear to me how far he intends to go 
   beyond simple markup to change fonts (a la HTML presentation tags). If it
   uses Pango, hopefully the other stuff will come along for free. We'll see.
   For Western (simple) scripts, automatic ligature support would be wonderful,
   but we need to be able to suppress selected ligatures (e.g., 'ff' in the
   English word 'shelfful'). Support for swashes and alternate glyph choices 
   would be very nice to have (embedded markup language?). For complex scripts 
   like Arabic family and southern/southeastern Asian families, proper support 
   (Pango?) is vital.

   UPDATE: See Text::Layout and HarfBuzz::Shaper packages. Layout is usable
     with Builder (but no explicit support yet). Shaper is supported by Builder
     for ligatures and complex scripts.

B. Unification of font support: including character set and encoding support
   improvements [see CTS 16 and CTS 23] to make more commonality between using 
   UTF-8 and single byte encodings, across all the font types (core, TrueType, 
   Type1/PS, etc.). One problem with core fonts is, even though most core fonts 
   are already TrueType, that only the Latin-1 glyph set has widths defined, 
   and only single byte encodings are possible (similar for Type1/PS fonts). To 
   support UTF-8 for core and PS, the font might have to be built on the fly 
   for a page (like a synthetic font), with translations to single bytes for 
   all glyphs. If the resulting font exceeds 256 characters, something would 
   have to be done to split the page internally into two or more sections, 
   each with their own embedded virtual font. Glyph widths would have to be 
   available for all characters.

C. Improved documentation, possibly even a book giving detailed explanations
   and examples, as both a reference and a tutorial. Needless to say, there
   would have to be sufficient interest to warrant the time and expense of
   writing/editing and publishing (in any format) a book to be sold!

D. PDF/A (archival document management, RT 120375): this might be more than
   throwing a few flags/overriding flags to force font embedding and no 
   encryption/ passwords. There may be other stuff that needs to be done to 
   achieve recognition as a proper archival format (and there are apparently 
   several archival formats).

E. JPEG2000 image file support (CTS 12): I don't know if this is worth it, as
   there seems to be very little use of this, but if someone is interested,
   have at it...

F. Fix Bar Code generation (CTS 1): there seems to be something quite wrong
   with the current bar code generation, so it's possible that no one is using
   it in real documents yet. It's also possible that I'm not writing my test
   cases properly -- does anyone know if they work? I suspect that the use of 
   XForms (relocatable text and graphics) for the bar image is not scaling 
   nicely, and may have to be replaced by drawn graphics primitives (text and 
   graphics drawn in their final place). Many other 1D and 2D bar codes 
   (including QR) would be good, but perhaps the bar codes should go into a 
   separate module, due to their potentially large code size and use of "new"
   Perl modules. Even the existing 5 or so formats could be moved out, as 
   presumably no one is using them yet (if they are, in fact, broken). This is 
   in section I, as bar codes are already implemented in the base, but it's 
   possible that bar codes could be removed and reimplemented in section II as 
   a separate library or module.

G. Fix Small Caps (and capitalization in general) for ligatures (CTS 13):
   some ligatures given in Unicode or single byte encodings don't get properly
   uppercased. The probable solution would be to decompose ligatures to their
   individual letters before capitalization or Small/Petite Caps (if an 
   uppercase version doesn't exist in the font, or use GSUB processing to 
   recreate a ligature from the capital letters). As Perl doesn't seem to handle
   capitalizing ligatures properly, a "capitals" function would need to be 
   offered, as well as improvements to the Small Caps in "synfonts". Various
   non-Latin single characters (e.g., Greek terminal/nonterminal sigma, German
   eszett, long s) also may need proper handling for capitalization.

   UPDATE: It may be better to use individual letters (rather than ligature
     Unicode points), allowing easy capitalization and small caps. Then use
     HarfBuzz::Shaper to replace lowercase letter sequences with true ligatures
     on the fly.

H. Fallback glyphs (CTS 5) when a desired glyph is not found in one font,
   but can be found in another. This is similar to HTML when you give a font
   family list in CSS. Pango might help with this.

   UPDATE: This is being considered for Text::Layout, but nothing scheduled yet.

I. Support for tagged structure (CTS 17 and RT 120375). At least, don't corrupt
   an existing tagged PDF file when extracting pages.

J. Adding comment fields to any object (and possibly standalone comments as
   their own objects). An example would be an image object with a comment
   giving the source image file, for debugging purposes).

K. Text method to move to arbitrary points: relative or absolute movement
   horizontally and vertically (a range of units), including tab support 
   (including \t and \v embedded in text), and maybe \n while we're at it.

   Note that tabs bring up some issues. First, a tab by character count (the
   traditional way, e.g., to the next n8-th column) is useful only for 
   monospaced fonts, and no changes in font size in the line. Thus, tab stops 
   would be more useful when defined by some absolute dimension (e.g., inches 
   or ems) of column position. Second, tabbing is usually done to get text 
   columns (sub columns), which involves a lot of manual setup and twiddling of 
   text. Consider using a TABLE within the column or page to get text organized 
   into the desired format (see "tbl" addition in section II).

   UPDATE: The $text->distance() call permits arbitrary delta-x and delta-y
     (in points) of text movement. It's not hard to convert non-points to 
     points, such as distance(2.4/in, -1.23/cm). For tables, take a look at
     the PDF::Table package for now.

L. Determine what it is about "CJK" fonts (.ttf and .otf) that makes them
   incompatible with synfont [RT 130040] and embedding [RT 130041], and fix if 
   possible. Are separate CJK fonts even necessary these days?

M. Add decorative rectangular box effects around sections of text. With or
   without border (allow rounded corners) and background color, drop shadows
   (3D effect), etc. The box is drawn at given dimensions and location, and
   the text written over it in the usual manner. Content clipping might also
   be supported.

I. Extend HarfBuzz::Shaper use (see A.) to flow paragraphs and sections (fill)
   to match capability of existing text-fill calls. Architect so as to extend
   easily to full paragraph shaping and "pouring" text into arbitrary columns,
   with balancing. Justification to avoid ragged-left or -right needs to be
   handled carefully for connected glyphs (e.g., Arabic, Indic, cursive Latin).

=============================================================================
II. Items to add to a separate area (new module or sub-module)

These are things that not everyone will require, and so should be split out 
into possibly a separate module (dependent on PDF::Builder). Some of these
things are getting into the realm of support for markup languages and word
processing.

A. Hyphenation and paragraph shaping: including CTS 20 (Hyphenation) and CTS 24 
   (pseudo page objects). The idea is to use Knuth et al.'s line-splitting and 
   paragraph shaping algorithms to flow text into a space in a visually 
   pleasing manner, while obeying widows and orphans constraints (as well as
   not orphaning headings). Pango may help here with line and word splitting.

B. Virtual pages: this would be related to item (A) (paragraph shaping), where
   PDF code would not be immediately written to an output page, but would be
   buffered, and output only later. This permits easier paragraph shaping and
   other rearrangements across columns and pages, where the starting location
   of a line of text is in the buffer, and it can be updated when moving the
   line around. Even individual words might be tagged (location and hyphenation
   points) so that lines could be broken at will. Even a limited amount of
   virtuality (virtual line output) could be useful for resetting a baseline
   to accommodate a change in font size -- this might involve tagging a word or
   block of words of the same height.

C. General text flowing capability, to fill irregularly shaped columns (such
   as with intruding inserts or margin notes) in a balanced manner, including
   spanning headings across all columns, where appropriate. This would also
   include flowing text around images, tables, or other inserts to avoid
   leaving large empty sections of pages (e.g., have a large table that floats
   to the next page, with text after it that could easily come before it on
   the original page). Something to handle cross references would be handy 
   here, to output "see table X above" or "below", "on the previous page"/"on 
   the next page", "on page X", etc. in a prescribed and consistent manner.
   Note that it might be good to notify the user during processing that such a 
   move has been done, so that it can be inspected.

   UPDATE: Columns could be any shape, drawn with lines, polylines, arcs,
     circles, splines, etc. Text baselines don't necessarily have to be 
     horizontal. Clip baselines to the "column" shape to get the line length
     and starting point for each line of text. There may have to be some 
     iterations to reshape a line if its height results in a shift of the
     baseline into a wider or narrower area.

D. Font Families: per CTS 22, make it easier to deal with switching fonts
   and variations within a font (bold, italic, size, color, underline, small
   caps, etc.), possibly with HTML tags inlined. The idea would be to only have 
   to specify a typeface and initial size, and then switch in and out of 
   variants (bold, italic, etc.) without having to call all the font routines 
   yourself. Perhaps several formats of markup could be supported (HTML, 
   Markdown, troff), driven by a definition file? Pango may help with this, at
   least with font-specification markup.

   UPDATE: Text::Layout package may prove very useful here, although it needs
     some enhancements (a new "back end" to return text for shaping, rather 
     than directly outputting it).

E. Continuing (D), eventually much of HTML and other common markups (headings, 
   quotes, HTML entities, tables, lists, etc.) supported. One goal would be to
   eventually support enough of each markup to have a separate converter 
   product (HTMLtoPDF, troffToPDF, etc.), but support for full Javascript and
   CSS (for HTML pages) will be a bear! Some level of macros (predefined
   strings) would be useful. Non-HTML might be converted to HTML or Pango
   format first.

F. Support for SVG graphics (drawing), support for troff's eqn, pic, and tbl
   markup languages to make it easier to do anything other than plain text.
   LaTeX equation and table handling would be good to have, too, to avoid 
   having to rewrite marked up text. Also provide a full graphing functions 
   library (stacked/unstacked line, bar, scatter plots etc. in 2D and 3D).

G. Prepress production markup: convenience functions to place a watermark or
   draft notice on all (or selected) pages, crop marks (based on trimbox),
   temporarily draw page bounding boxes, temporarily draw object limit boxes,
   color dots/bars for color printing alignment, instructions to the (human)
   printer.

   Page background color or pattern should extend to the full size of the page
   and not end when content ends part way down the page. Remind users that 
   most printers will not print all the way to the edge.

=============================================================================
