Repository navigation
getRawTextContent does not returns words' pieces in proper coordinate order #421
Description
Hey,
getRawTextContent seems to assume that word pieces in the bidiTexts array appear in ascending order of their y coordinate but they probably do appear as they are being added by the PDF rendering software. I have a file where a line of text looking on screen like
Some label 12.3 4455
appears in that array as
Some label | 3 | . | 2 | 1 | 5 | 5 | 4 | 4
that is, the character groups are properly following one after the other but the order of individual characters within the groups is reversed. And getRawTextContent() naively returns
Some label 3.21 5544
The y number of each item correctly shows the correct position but the code is not considering that (it only checks the absolute y difference).
So I wonder if this has a known solution before I go and change the code myself?
Cheers for the nice work.
Activity
modesty commentedon Mar 1, 2026
can you upload the PDF?
gkostov commentedon Mar 1, 2026
There is personal identification information in there so can't upload here. May I send it to your email (that's on your Github account)? Thanks for your help!
gkostov commentedon Mar 1, 2026
Hm, just checked again and it seems the problem is not in
getRawTextContentdirectly because thebidiTextsalready has those characters concatenated. I saw the raw character elements appearing inpdfData.Pages[0].Texts. And even if I've not yet seen that array to be processed somewhere theyvalues in there indicate the correct ordering of the characters. I need to trace the code more to see where exactly the concatenation goes wrong.It seems
pdf2json/base/core/evaluator.js
Line 808 in 48b50bf
bidiTextsarray and it added characters in order of appearance from the parser of the binary stream.modesty commentedon Mar 1, 2026
you can email me or create new PDF, either way works.
tanin47 commentedon Apr 28, 2026
I can provide an example. The PDF was made with Google Doc.
getRawTextContent()returns the line in the reversed ordersample.pdf
gkostov commentedon May 3, 2026
I saw the new sorting method added with b0067d7 , which is also available in the 4.0.3 release. I am curious if this commit is related to the present issue (it does not solve it because it is sorting the
bidiTextsarray)?gkostov commentedon May 4, 2026
Been chasing this again and I found that the
showSpacedTexthandler herepdf2json/base/display/canvas.js
Line 1334 in 8554a3a
.xcoordinate of each character from right to left. I still don't know why this is so but it does a good job. While the "TJ" handler herepdf2json/base/core/evaluator.js
Line 901 in 8554a3a
showSpacedText) simply concatenates the characters so they progress to the right. Unfortunately the simple swap ofchunk += fontCharsToUnicode(items[j], font)withchunk = fontCharsToUnicode(items[j], font) + chunkdoesn't work great - it produces those numbers in the correct order but reverses the order of words. The result is like:I think this is a step closer to the solution but I need to leave it for now.
Cheers