====== PDF Survey Annual School Census ======

===== REST API endpoints =====

The Rest interface has these methods. Currently, they can only be access directly from the browser address bar:

This repairs some structural issues in the template. This was already done and is no longer needed.

<code>
/api/pdfSurvey/fix/<<year>>?version=<<SEC|PRI>>
</code>

Take a survey, reload its data into the current version of the form. This is what is needed primarily to get data from previously loaded (using flawed PDF survey) and reload it into a fixed PDF survey.

<code>
/api/pdfSurvey/reload/<<schno>>/<<surveyYear>>?TargetYear
</code>
 
Create a new survey using the survey Year template, reading the data from the database.

<code>
/api/pdfSurvey/generate/<<schno>>/<<surveyYear>> 
</code>

===== PDF Intricacies Notes and Primer =====

<note tip>The following is a rather technical discussion that arose due to some visual issues we encountered in the legacy PDF survey templates when trying to correct a flaw in it. It is unlikely to be knowledge commonly used and needed in the future but is kept here for historical purposes and it may perhaps be useful.</note>

These two objects are pair of OnStaff checkboxes: 

<code>
15772 0 obj

<</AA<<>>/AP<</D<</Off 18786 0 R/Y 18784 0 R>>/N<</Off 18783 0 R/Y 18781 0 R>>>>/AS/Y/BS<</S/S/W 1>>/F 4/MK<</BC[0.50196 0.50196 0.50196]/CA(4)>>/P 4861 0 R/Parent 14778 0 R/Rect[298.05 392.45 311.05 405.45]/Subtype/Widget/Type/Annot>>

endobj
</code>

<code>
15773 0 obj

<</AP<</D<</N 18785 0 R/Off 18786 0 R>>/N<</N 18782 0 R/Off 18783 0 R>>>>/AS/Off/BS<</S/S/W 1>>/F 4/MK<</BC[0.50196 0.50196 0.50196]/CA(7)>>/P 4861 0 R/Parent 14778 0 R/Rect[326.85 392.45 339.85 405.45]/Subtype/Widget/Type/Annot>>

endobj
</code>

the Property /AP is the appearance dictionary:

<code>
/AP<</D<</Off 18786 0 R/Y 18784 0 R>>/N<</Off 18783 0 R/Y 18781 0 R>>>>
</code>
 
It in turn has properties pointing to two subjects /D (down appearance) /N (normal appearance). Within those are the possible values for the checkbox: this one can be either Y or Off. The objects 18786 18794 18783 and 18781  are the actual drawing routines for these possibilities.

e.g. here’s 18781 which draws the green tick:

<code>
18781 0 obj
<</BBox[0 0 20 20]/Length 117/Resources<</Font<</F1 18780 0 R>>>>/Subtype/Form/Type/XObject>>stream
3 w
0.50196 0.50196 0.50196 RG
0.98828 0.95703 0.89844 rg
0 0 20 20 re
B
BT
/F1 18 Tf
0 0.50196 0 rg
4 4 Td
(4)Tj
ET
endstream
endobj
</code>

This AP dictionary is what is set for each Checkbox in the StandardiseCheckboxes code. 

Now if you choose the Preference option, these /AP entries are effectively ignored, and Acrobat (or whatever reader you are using that offers similar functionality) attempts to draw the checkbox using your preferred backcolor, and border.  What I’ve  identified tonight is that you can give it a hint by setting another dictionary on the widget: /MK (“Appearance Characteristics”) – if the MK is there Acrobat will use it.

In MK you can put /BC – border color and /CA – caption or text.

Here’s /MK for the Tick checkbox:
 
<code>
MK<</BC[0.50196 0.50196 0.50196]/CA(4)>>
</code>
 
The caption 4 is the Dingbat character for the Tick.

One thing you can’t set in /MK is the Text Color, so we don’t actually get a green tick anymore if using this “dynamic” rendering based on the preferences.

But, I have modified Standardise Checkboxes to create all these /MK dictionaries with /CA pointing to the right character for the given checkbox, and /BC set to grey.

The pair of checkboxes now look like this:

<code>
15772 0 obj

<</AA<<>>/AP<</D<</Off 18786 0 R/Y 18784 0 R>>/N<</Off 18783 0 R/Y 18781 0 R>>>>/AS/Y/BS<</S/S/W 1>>/F 4/MK<</BC[0.50196 0.50196 0.50196]/CA(4)>>/P 4861 0 R/Parent 14778 0 R/Rect[298.05 392.45 311.05 405.45]/Subtype/Widget/Type/Annot>>

endobj
</code>

<code>
15773 0 obj

<</AP<</D<</N 18785 0 R/Off 18786 0 R>>/N<</N 18782 0 R/Off 18783 0 R>>>>/AS/Off/BS<</S/S/W 1>>/F 4/MK<</BC[0.50196 0.50196 0.50196]/CA(7)>>/P 4861 0 R/Parent 14778 0 R/Rect[326.85 392.45 339.85 405.45]/Subtype/Widget/Type/Annot>>

endobj
</code>
 
You can tell the N one from the Y by looking at:

  * The keys in the /AP /N dictionary ( you see either /Y or/ N)
  * The CA value in the /MK dictionary ( either 4 (tick) or 7 (cross)

This is a code change so you’ll need a code update to get this to work on your machine. 

One other thing, itext allows you to save PDF with no compression so most of it (except images) is plain text. This make this level of debugging possible.

So the code you identified does set the Appearance Dictionary of each box, and these render correctly except when the Preferences force a “dynamic” render.   

In that circumstance, Acrobat basically ignores the /AP rendering code, and tries to build its own renderer.

Prior to the changes last night, it would get the wrong icon for this, and you may be right – it takes the icon associated with the first of the two child widgets linked to the field.

This linkage fyi goes in both directions – via the parent element on the widget, which points to the field:

{{ :annual-census-user-manual:pdf-internals-1.png?nolink&400 |}} 

And via the Kids array of the field, which points to the widgets.

{{ :annual-census-user-manual:pdf-internals-2.png?nolink&400 |}} 

So with nothing better to go on, Acrobat was looking at the first Kid, and this was here it was making mistakes.

But now, when constructing the “dynamic” rendering, it can use the /MK Appearances Characteristics dictionary, to get the symbol, and border color:

{{ :annual-census-user-manual:pdf-internals-3.png?nolink&400 |}} 

With this information, it contructs the “dynamic” rendering pretty close to the customised rendering. (except for the Text color).

Small point, “dynamic rendering” is only used on the ‘Normal View’ of the widget, the Down view (ie mouse down) still uses the /D appearance in the /AP dictionary; this is why you get the darker grey background when the mouse is pressed. Also the Focus appearance uses the /N Normal appearance from the /AP dictionary even when “dynamic rendering” is on: this is why you’ll see a tick go green when you click, it, when it loses focus it turns black (ie swaps to the dynamic rendering.

In Acrobat the #0 #1 Is just a shorthand to enumerate the Kids array during editing. Its not recorded in the PDF. IT seems that it reflects the order in the Kids array:

{{ :annual-census-user-manual:pdf-internals-4.png?nolink&400 |}}  

In the pair 15816 15817 its 15816 that is N:

{{ :annual-census-user-manual:pdf-internals-5.png?nolink&400 |}}  

The parent 14808 of this OnStaff field is:

{{ :annual-census-user-manual:pdf-internals-6.png?nolink&400 |}}  

Ie that field is TL.04.OnStaff

But this ordering has no impact on the data collection as the expert values Y and N are correctly set on the left and right widgets. It can make a difference to the rendering, but the change I made last night to add /MK information is probably more robust.