Gökçen Eraslan - Yere Düşünce Derneği

GSoC Final

2012-08-24T00:00:00-07:00

After the GSoC midterm, there were two remaining important objectives to do: to write the NSS code to create a PKCS7 object (which includes the actual signature, encrypted SHA1 digest and the public certificate) and to improve the PDF signing GUI (which can be tested only in the experimental mode).

For the NSS part, first I have used the NSS PKCS7 API (secpkcs7.h and pkcs7t.h) to create the PKCS7 object. But it was very hard for me since NSS lacks a proper documentation (the one in the header files is not sufficient). Anyway, later on, I have decided to use the new and more fine grained NSS CMS (Cryptographic Message Syntax) API (cms.h and cmst.h).

I have followed following steps to create a PKCS7 object:

We need a NSS CERTCertificate structure to operate on the NSS world. So, DER encoded certificate data is obtained via the Encoded attribute of XCertificate and converted to a CERTCertificate structure using the CERT_DecodeCertFromPackage function.
SHA1 sum of the PDF file is computed using the HASH_Create, HASH_Begin, HASH_Update and HASH_End functions.
PKCS7 object is created:
1. An empty CMS message is created using NSS_CMSMessage_Create.
2. A SignedData is created inside the CMS message using the NSS_CMSSignedData_Create function.
3. Since we use a detached PKCS7 object (which means that the PKCS7 signature object contains only the signature part, excluding the data itself) for the signing, an empty Data object is created inside the SignedData using NSS_CMSContentInfo_SetContent_Data.
4. Certificate chain and signer info is added to the SignedData, using NSS_CMSSignerInfo_IncludeCerts and NSS_CMSSignedData_AddSignerInfo respectively.
5. SHA1 is added to the CMS message using NSS_CMSSignedData_SetDigestValue.
6. CMS message is DER-encoded via NSS_CMSEncoder_Start and NSS_CMSEncoder_Finish. (NSS_CMSEncoder_Update is not used since we don’t have a Data).
Finally, DER encoded PKCS object is converted to the HEX representation and written to the relevant structure of the PDF which is prepared in the earlier commits.

And for the GUI part, I have used Thunderbird certificate selection GUI design in the PDF export dialog. Now, the selected certificate information is printed in a disabled Edit control and users are able to clear the selected certificate to cancel the signing operation. (I have also used the PassWord = True property in the .src file for the certificate password input.)

At the end of the day, however, Adobe acroread shows the PDF signatures as invalid for some reason. But, it shows the certificate details correctly (which was not the case when I have used the old NSS PKCS7 API). So it seems DER/HEX encodings and the inclusion of the certificate chain are correct but there is a bug, apparently. Maybe the calculation of SHA1 or passing the private key password to NSS is wrong. I will try to figure it out soon.

Thanks to kendy and sberg for their devoted support in this GSoC project.

The first half of GSoC 2012 is over

2012-07-21T00:00:00-07:00

The first half of the GSoC 2012 is over now. Since LibreOffice team set ‘merging GSoC feature branch into master’ as a requirement for the midterm, I have marked PDF Signing fature as experimental feature and merged into master¹, and recently I have deleted my feature branch. From now on, I’ll continue to work on the master.

All changes I have made can be seen in the merge commit but let me explain what those commits mean in detail.

First, I have added the Signature widget annotation type to vcl/inc/vcl/pdfwriter.hxx and vcl/source/gdi/pdfwriter_impl.cxx.
emitSignature and finalizeSignature methods are added to PDFWriterImpl class. Those methods are called in PDFWriterImpl::emit() which is the final operation of PDF creation, it writes all the prepared PDF structures to the given file in order. Page, resource, outline, catalog, annotations, metadata objects are all written to the PDF file in the emit method.

At the beginning of the emit method, I have checked the PDFWriterContext to see if signing is requested² and then create a Signature annotation object. Next, after the Catalog object is emitted, the main Signature objects (/Type /Sig and /Type /SigRef) are emitted. But, unfortunately we have to include a byte range in the signature dictionary as well as the digest of that byte range. Since the document is not finalized yet (the last part is the Trailer in the PDF file.), we cannot specify a ByteRange and the document digest now. Instead, we write a dummy ByteRange value and a dummy signature and reserve a space to fill it later when the document is finalized. Of course, we have to write down the offset values of that dummy byte range and digest object, so we can fill them later on.

Finally, when the trailer is emitted in the ::emit method, we are ready to fix the ByteRange values and include the digest value. This is what the finalizeSignature method does. finalizeSignature calculates the real ByteRange value and fix the old values. Next, reads the whole PDF data to create a PKCS7 object using NSS library. But since we don’t have a certificate now (PDF export dialog has to be changed to make users select the certificate) PKCS7 object creation is not implemented.
I have added a new Digital Signatures tab to the PDF export dialog³. But this wasn’t easy. First I thought that certificate selection dialog is used in document signing and it must be easy to re-use it. But, that dialog (xmlsecurity/source/dialogs/certificatechooser.cxx) seems to be an internal part of the xmlsecurity module and can only used in the signDocumentContent method of the XDocumentDigitalSignatures UNO interface. So, I had to extend that UNO interface and added a chooseCertificate method. It just asks user to choose a certificate and returns it. That’s all. So, certificate chooser dialog can now be used in anywhere using the com::sun::star::security::XCertificate chooseCertificate();.

So, to finalize the signed PDF document now I have:

the document data to compute its digest,
DER encoded certificate of the user,
the password of the user to decrypt his private key and sign the digest.

Also, I now the byte offset of digital signature to write the computed PKCS7 object. I hope, adding a few lines will be enough to complete signing :)

Cheers.

By the way, since I didn’t use --enable-werror configure option and ignored some warnings caused by my changes, I have broken some builds in the master tinderbox. Sorry about that :) Now I know, I must use --enable-werror and --enable-dbgutil all the time.
↩
As you know, PDF export dialog has a great deal of options. This dialog, which resides in filter/source/pdf/impdialog.cxx, basically takes all PDF export parameters and passes them to PDFFilter class and it forwards those parameters to PDFExport class. Finally, PDFExport class instantiate a PDFWriterContext object and pass it to the vcl::PDFWriter class. (I want to thank GDB to make this process easier to understand.)
↩
Right now, password box is an ordinary text edit. So be careful, your private key password will be exposed :) As a second matter, the design of the certificate selection is ugly. I may use the certificate selection design of Thunderbird. There are disabled text edit, a select and clear button. Thus, I can also remove Sign PDF File checkbox.
↩

PDF signing GSoC roadmap

2012-06-08T00:00:00-07:00

Actually, I’ve sent a mail about that roadmap but for those who didn’t see it, here is the current progress in GSoC PDF signing project.

After weeks of reading the PDF reference and inspecting the PortableSigner implementation, here is what I must do to implement signing support:

Create an Annotation object of Widget subtype. Make it’s width and height zero since we don’t want to make the signature visible, for now. (Yes, digital signatures may be visible. There is even a document about that.)
Create and AcroForm field (/FT/Sig) as the parent of the Annotation object we created. Only buttons, combobox, listbox, edit and hierarchy is defined now.
Create a Signature dictionary (/Type/Sig). AcroForm field must point to that dictionary in the /V field. This is the crucial object. Prepare a PKCS#7 object and use it as the /Contents value. It must contain the X.509 certificate and the encrypted message digest.
Modify /Catalog object of the PDF document so that /AcroForm points to the our AcroForm field.
Modify the first /Page object so that it’s /Annots key points to our Annotation object.
Write a new xref table and the final trailer.

That’s all, it’s very easy :) It seems most of these changes will be applied to vcl/source/gdi/pdfwriter_impl.{c,h}xx files.

As Thorsten said, I will mimic the behaviour of the PortableSigner tool first. PortableSigner use the incremental updating feature of the PDF. This feature allows pdf editing tools to edit PDF file by adding new or existing objects to the end of the file. So, if an object exists more than once, last occurence is used.

In the PDF export code I have to:

Add PDFWriter::SignatureWidget AcroForm type and use PDFWriterImpl::createControl() to add a Signature field.
Now, PDFWriterImpl::emitCatalog() method writes all the Pages, Resources and Catalog objects. I think I need to split those into different methods, so I can emit the modified catalog again.
Re-write the /Page object that contains recently added Signature annotation.
Add a new xref table and a new trailer at the end. I don’t know how to implement that now. PDFWriterImpl::emitTrailer() method writes all objects but in incremental updates we must include only added/modified objects to the xref table.

I know that is a kind of “note-to-self” but I’ve tried to write down all the steps that I can think of.

A Short Introduction to PDF

2012-05-20T00:00:00-07:00

In this blog post, I’ll try to summarize what I have learnt about PDF so far. I used the ISO 32000-1:2008 document as a reference.

Standardization

To make desktop printers able to render complex text and graphics files in a uniform way, PostScript page description language is created in 1985. Similarly, as a way to view these complex text and graphics files electronically in a platform independent manner, PDF is created by Adobe Systems company in 1992 under the name Project Camelot.
PDF was originally a proprietary format controlled by Adobe. In 1993, the first complete PDF specification is published (PDF 1.0) by Adobe and this specification is free of charge since 2001. The last reference is the PDF 1.7 file issued by Adobe.
There are also some specialized and more restrictive subsets of PDF such as PDF/A (PDF for archive), PDF/X (PDF for exchange), PDF/E (PDF for engineering) etc. These subsets are released as ISO standards in 2001, 2005 and 2008 respectively. As I said, those are more restrictive file formats. For example, you may embed the fonts into a PDF file or not (more on that later). You are free to use external references for fonts, images, videos etc. (just like the HTML files). But if you are preparing a PDF/A compatible PDF file, you have to embed all the fonts you used. In PDF/A, the external content references are forbidden as well as audio/video content and encyption. Shortly; in these subsets, some PDF features that are not suited for some specific reasons (like archiving, exchange etc.) are left out.
Since 2008, PDF is an ISO standard, namely ISO 32000-1:2008. This standard is consistent with the PDF 1.7 which was the last PDF specification published by Adobe. Future versions of PDF will be published by the ISO technical commitees. But after ISO 32000-1, Adobe published some extensions to the ISO 32000-1:2008 standard and those extensions will be added in ISO 32000-2 which will be the PDF 2.0.

PDF internals

Now, I want to mention about some interesting bits of the PDF:

PDF files start with the %PDF-x.y marker stating which version of PDF the file comforms with. Even though the rest of the file is binary, all the keywords of the format (such as obj/endobj, stream/endstream or Named Objects like /Type, Length etc.) are ASCII strings. So, you can roughly have an idea about the structure of the document when you inspect the file with a text editor. But be careful about file editing! Since text editors may change/convert line endings, it’s not recommended to edit PDF files with text editors.
Type 1, TrueType and OpenType fonts can be embedded to PDF files. If you want the document to be smaller, subset of a font may also be embedded. But whether you embed any fonts or not, PDF standard assumes that viewer applications can display 14 basic fonts such as Helvetica, Times, Courier etc. In the case that you refer to a non-standard font that is also not embedded, it’s the responsibility of the viewer application to find those fonts in the environment. As you guess, this approach damages portability since users may not have the fonts used in the file.
All the text, images, numbers, shapes and even the pages of the PDF document are represented as “objects” in PDF files. At the end of the each PDF file, there is a section called xref (read as the “cross reference table”), that keeps the byte offset of the objects from the start of the file to provide random access to all objects. xref is stored at the end of the file, so that the PDF creator applications can create a PDF file in a single pass. This also provides an efficient way to view the document, for example think about a document with thousands of pages, if a reader wants to render page 456, it can find that page quickly using the xref table. PDF files are read from the end of the file to get the xref table and object offsets.
Encryption and digital signing is possible in PDF. For password-based symmetric encryption; RC4 (proprietary algorithm of RSA) or AES (starting with PDF 1.6) algorithms are used with 40 or 128 bit keys. Actually, MD5 digest of the password is used as the key of RC4 or AES. Additionally, public key encyrption can used used with the combination of symmetric encryption. Basically; symmetric key that is used to encrypt the document is encrypted with the public key of a user. When that user gets the document, she first decrypts the symmetric key using her private key and then decrypts the document with the symmetric key. Moreover, you can define user-based access permissions such as user Ibrahim cannot print the document but can fill the forms, user Ozan can add annotations but cannot copy text etc. (I will tell the details about the digital signatures in the next post.)
PDF files may contain interactive forms fields (such as push buttons, radio buttons, checkboxes etc.) to gather information interactively from the user. You can also define an Action and associate that action with a form element (like a button). So it’s possible to play a sound, submit form values to a URL, reset form fields, interpret JavaScript code, launch an application etc. when a button is pressed. I know, JavaScript seems interesting, but for example it’s possible to modify the appearence of the PDF file using JavaScript. Here is the JavaScript API defined by Adobe.
Compression of the objects (text, images etc.) are supported. So, you can compress your images using JPEG, JPEG2000, CCITT and JBIG2 while you can use LZW or Flate for text, graphics or images.
Incremental updates: PDF allows modifications to be appended to the end of the file. This appended part only contains the modified objects in the PDF file. In this way, you don’t have to load the whole document and rewrite all parts of the document each time it’s saved. For example, when you digitally sign the document using a tool (such as PortableSigner, you can see that only some new objects including /Sig and /SigRef dictionaries are stored at the end of the file.

Another advantage of this method is the possibility of undoing changes. Since the original document is preserved, it’s easy to render it.

That’s it for today.

GSoC 2012 - Introduction

2012-05-14T00:00:00-07:00

A Short Introduction

Firstly, let me introduce myself. My name is Gökçen Eraslan, I’m a FLOSS developer and Computer Science M.S. student in Bogazici University of Turkey. Previously I worked for TUBITAK (The Scientific and Technological Research Council of Turkey, equivalent of NSF in Turkey) for 4 years. I was a developer and the release manager of the Pardus Linux distribution project which is a government funded project aiming to develop a mature Linux distribution along with technological innovations (a brand new package manager, installer, configuration system etc.) as well as encouraging free software development in Turkey in order to avoid the massive sums needed for the license fees of proprietary operating systems. But lately Pardus project is somehow suspended (details here) and most of the core developers left Pardus.

GSoC 2012

I have been selected as one of the GSoC 2012 students of the LibreOffice project. My GSoC proposal is to add digital signing support to PDF export feature. This is an idea from The Document Foundation ideas page. Although I’ve fixed a python-based LibreOffice easyhack, I’m not experienced in LibreOffice codebase. But since the document signing (not PDF signing) is already supported by LibreOffice, I hope certificate management (such as listing and choosing certificates provided from the certificate database of Mozilla applications) will be easy using the existing classes.

Right now, I’m reading ISO 32000-1:2008 PDF standard and my next planned step is to be familiar with PKCS#1, PKCS#7 and PKCS#12 standards and to learn code pointers to PDF export code in LibreOffice.

Tools and Frameworks for PDF Signing in Linux

I am also looking for the ways of signing PDF files digitally in Linux. So, I can look at some real world examples about PDF signing and compare the documents signed by LibreOffice and those tools. Here are popular PDF signing tools and frameworks that I found:

PortableSigner: Best known PDF signing tool in Linux which is written in Java and licensed under EUPL. PortableSigner provides both CLI and GUI tools and requires your PKCS#12 file (containing your private key) and an X.509 certificate file (that authorizes you) to sign a PDF file. (In my next blog entry I want to briefly explain how digital signatures work.)
iText: Best known PDF framework also written in Java and licensed under AGPL. There is a C# port called iTextSharp as well. This framework implements various classes to generate PDF files as well as the classes to update PDF files. So it’s can split, merge, sign and encrypt PDF files. Actually, iText is the crucial component of many known PDF tools including PortableSigner, pdftk etc.
Proprietary tools such as Adobe Reader, myPDFSigner or PDF Studio Pro can also sign PDF files. I tried Adobe Reader but I couldn’t make it work under Linux due to the 32-64 bit library problems. (It requires 32-bit libxul.so and crashes with SIGSEGV when I point it to that library.) Anyway, I think the only benefit of Adobe Reader will be that I can use it to verify my signed PDF files.

That’s it for today.