Not Enough RAM When Using Adobe Acrobat OCR

[krs]

Note: New thread created since discussion became lengthy in other thread.

On the subject of RAM....

I just ran into an issue of not enough RAM when using Adobe Acrobat OCR for a large document.
I currently use a 2012 Mini with 16GB of RAM an a 1 TB SSD
When I run OCR on a 160 page document, I get an error message at around page 70 that there is insufficient memory.
Seems that RAM swap with the SSD can't be used/is not supported.
Talked to a friend who does a lot of Arobat OCR work (who does not have that issue on large documents) turns out that he has 64MB of RAM.
Now I'm wondering if anyone with a Mac with an M1 or M2 chip is using Acrobat OCR on large documents and if there are similar problems of running out of RAM even with PAM and storage being integrated on those Mas in a sense.

BTW - My workaround is to split the pdf into 70 page chunks, running OCR on each section and then combining the 70 page OCR sections.

 

[pigoo3]

krs said:
Now I'm wondering if anyone with a Mac with an M1 or M2 chip is using Acrobat OCR on large documents and if there are similar problems of running out of RAM even with PAM and storage being integrated on those Mas in a sense.

A 2012 Mini with 16GB RAM & Intel CPU (and friend with Mac-Mini & 64GB RAM)...and a 2020 M1 Mac-Mini with 8GB RAM...are totally different animals. RAM is managed way differently between Mac's with an Intel CPU & Apple silicon.

One of the most demanding things we can ask a computer to do is video editing:

* Many YouTube videos out there that compare video editing on a 2018 Intel Mac-Mini vs. M1 Mac-Mini's (M1 Mac-Mini with 8GB RAM blows away the 2018 Intel Mac-Mini).

* And YouTube videos that compare M1 Mac-Mini with 8GB Ram vs. M1 Mac-Mini with 16GB RAM...and video editing. M1 Mac-Mini with 8GB is fine for almost all/most users. M1 Mac-Mini with 16GB RAM possibly more necessary for professional level work.

Check out some YouTube videos to verify.

Nick

p.s. Sounds like more RAM in your 2012 Mac-Mini could/would help out with things. But comparisons to M1 Mac-Mini not really comparable (two models much different).

 

[MacInWin]

When I run OCR on a 160 page document, I get an error message at around page 70 that there is insufficient memory.
Seems that RAM swap with the SSD can't be used/is not supported.

I would bet that your storage area (hard drive) is almost full. The error you are seeing is generally caused by the drive getting so full there is no space for the swap file for that large document processing to be written out. Free up space on the drive and your process may very well work just fine. RAM swap with SSD works just fine, as long as there is space. For an SSD, it's best to keep free space as large as you can because SSDs start to slow down at about 50% full (it has to do with the internals of the way SSDs work, nothing from Apple).

 

[krs]

MacInWin said:
I would bet that your storage area (hard drive) is almost full. The error you are seeing is generally caused by the drive getting so full there is no space for the swap file for that large document processing to be written out. Free up space on the drive and your process may very well work just fine. RAM swap with SSD works just fine, as long as there is space. For an SSD, it's best to keep free space as large as you can because SSDs start to slow down at about 50% full (it has to do with the internals of the way SSDs work, nothing from Apple).

Well Jake, I would have won that bet.
I obviously checked the space on the SSD before I posted here - I have just over 329 GB free on the SSD.
RAM swap normally works fine but specifically for OCR with Acrobat there is some issue.

 

[MacInWin]

I think it would been a "push" on the bet.

Having 329GB of a 1TB SSD is pushing the limit. It's more than 50% full, which is where performance starts to fall off. Yes, it's not "almost full" but it is beyond what I would recommend. Throttle that back to get 500 GB free and try the OCR again. I don't know about Acrobat, and OCR, so maybe there is something unique there, but if the error is about running out of application storage, that points to the SSD, even if you start with 329GB.

On my 1TB storage, I am, right now, at 400GB used, 600 GB free, and I like to stay in that 400-500 GB range.

Jake

 

[krs]

pigoo3 said:
A 2012 Mini with 16GB RAM & Intel CPU (and friend with Mac-Mini & 64GB RAM)...and a 2020 M1 Mac-Mini with 8GB RAM...are totally different animals. RAM is managed way differently between Mac's with an Intel CPU & Apple silicon.

One of the most demanding things we can ask a computer to do is video editing:

* Many YouTube videos out there that compare video editing on a 2018 Intel Mac-Mini vs. M1 Mac-Mini's (M1 Mac-Mini with 8GB RAM blows away the 2018 Intel Mac-Mini).

* And YouTube videos that compare M1 Mac-Mini with 8GB Ram vs. M1 Mac-Mini with 16GB RAM...and video editing. M1 Mac-Mini with 8GB is fine for almost all/most users. M1 Mac-Mini with 16GB RAM possibly more necessary for professional level work.

Check out some YouTube videos to verify.

Nick

p.s. Sounds like more RAM in your 2012 Mac-Mini could/would help out with things. But comparisons to M1 Mac-Mini not really comparable (two models much different).
Click to expand...

Thanks but I know all that.
It seems that Acrobat OCR needs a large amount of RAM and memory swap is not an option.
I do a lot of video editing as well and I have never run out of RAM doing that.
Ideally, I would like to find someone with an M1 or M2 Mac with 8 GB of RAM who runs Adobe OCR on a 150 page or larger document.

 

[krs]

There is more discussion about this than I expected so I will start a separate thread tomorrow rather than tack more onto this one.
I would really like to understand what the issue is with Acrobat OCR; I can then also post a few screen shots and explain in more detail what is happening.

 

[Raz0rEdge]

There is more discussion about this than I expected so I will start a separate thread tomorrow rather than tack more onto this one.
I would really like to understand what the issue is with Acrobat OCR; I can then also post a few screen shots and explain in more detail what is happening.

No, don't do that. All discussing around the same topic should be kept in the same thread, we will merge related posts.

 

[krs]

I don't have much time right now to do any experimenting, like run OCR with a lot more drive space available to see if that is the issue, but here are a few more details which perhaps ring a bell with someone.
The original file was scanned directly as a pdf.
It's a catalog of 164 pages, the scanned copy (no OCR) was 38.3MB
When I run Acrobat OCR on that, I get to page 77 and then I get the Acrobat message, screenshot below. The critical part of the message is blank.
Initially I thought OCR had a problem with the table on that page, but it turned out that was not the issue.
When I continue to the next page after this Acrobat message, I get the same message again and that continues right to the end of the document.
At the very end I then get the message in the second screenshot, ie out of memory.
When I then check that document, I see OR performed up to page 77 and the rest are just images of the text.
If I then run OCR on page 70 and onward, everything works fine for another 70+ pages, so that table on page 77 was not the issue.
The end result of the complete 164 page document using Adobe Acrobat OCR was a 60MB document (done in three chunks)
I then ran OCR using PDFpen - that software had no issue and completed the OCR task (all 164 pages) but the end result was a 120MB file (which I did not use).

I still think the Acrobat OR software for some reason needs "real" RAM and can't use memory swapping with the SSD and based on that am wondering if there is a similar issue with an M1 or M2 Mac with only 8 GB of Ram.
I have used Acrobat for many other functions an it's only OCR where I see this issue.

 

[Rod]

Just my 10 cents worth but 329GB is dangerously close to the advisable 30% free storage for 1TB ie. 333GB.
I have a 512GB M1 MBP with 16GB RAM, an effective 494.38GB of storage with 284.27GB free.

 

[krs]

Just my 10 cents worth but 329GB is dangerously close to the advisable 30% free storage for 1TB ie. 333GB.

I don't think free drive space as a percentage of total drive space makes any sense with the large storage people now often get.
That was a reasonable way to define this years ago.
Two points come to mind:
1. If my drive had been a 250 GB drive, free drive space of 75 GB (30%) would be considered acceptable, but now when using a larger drive, one suddenly needs 258 GB of additional free space? Makes on sense to me.
2. A 2017 MacBook Air we use every day hasa 500 GB SSD with 40GB of free space and it runs just fine with no issues at all.

If one looks on the net,the recommendations vary all the way from a low of 5% to a high of 30%,with 10% a number often quoted.

But when I get some more time later this week, I will make a clone of my drive, delete enough files to get to 50% free space and run Acrobat Adobe again.

In the meantime I tried running OCR downgrading image resolution to 150 dpi from the default 600 dpi that I normally use.
This time I only got to page 70 (rather than page 77) and this message came up - at least the whole message showed this time.
Page 70 btw had very little text, mostly images.

 

[IWT]

I then ran OCR using PDFpen - that software had no issue and completed the OCR task (all 164 pages) but the end result was a 120MB file (which I did not use).

I totally understand the hike in MBs when you use PDFpen; but on the other hand it does the job swiftly, and completely; zero problems.

So, why not use PDFpen for your work and store the large files on an EHD?

Sounds reasonable?

Ian

 

[krs]

I totally understand the hike in MBs when you use PDFpen............

Could you expand on that?
Because I sure don't understand why PDFpen should end up with a file twice as large as Acrobat

So, why not use PDFpen for your work and store the large files on an EHD?

The files are meant to be downloadable from one of my websites so I try to keep them a reasonable size.
The original, non-searchable pdf file was 38.3MB which is fine; the searchable file using Acrobat at 60MB I think is already at the high end where I want to be with the file size of those documents, not any bigger.
I could try to reduce the size of the PDFpen document to 60MB or less and see where that leaves me as far as quality of the pdf doc is concerned.
The other issue with PDFpen is that I would have to spend about $150.- to get a license - this OCR test was done on a trial basis which adds a watermark to the pages.
If PFpen had given me an OCR document of 60MB or less, I would have spent the money.
I have to see which of the other Mac pdf applications Randy posted a while back also does OCR and see if they are any better.
Right now my workaround is to break up the original pf into 70-page chunks, run Acrobat OCR on each one and then combine them.
But I'm still trying to understand why memory swapping doesn't seem to work with Acrobat.
I'm thinking of watching RAM usage/swapping attempts in Activity Monitor when I get the next large pdf document to run OCR and see if that provides any hints.

 

[ferrarr]

Most Adobe products require using a scratch disk, is that an option with Acrobat OCR?


Sent from my iPad using Mac-Forums

 

[Randy B. Singer]

I don't think free drive space as a percentage of total drive space makes any sense with the large storage people now often get.

Yes, each time rotating disk hard drives have increased in size over the years, from tens of MB's, to tens of GB's, to multiple TB's, that's exactly what folks said. And each time they were wrong.

Each time that hard drives got bigger, the amount of free space that was required to keep your Mac running reliably didn't remain at a roughly fixed size. Instead it keep on increasing, and the roughly 20 to 30% rule of thumb remained constant.

It could vary quite a bit, though. It's a rule of thumb, not a hard and fast rule. The main applicable variable being how much disk fragmentation there was. A drive with extremely low fragmentation (drive fragmentation; the Mac defragments files automatically) could be filled up almost all the way. A drive with a lot of fragmentation might be for all intents and purposes full with as much as 40% free space remaining, no matter how big the drive was. Even if the drive was huge.

http://www.macattorney.com/ts.html
Item #5 and Note #1
Note the many citations.

Apparently as a drive gets bigger, the amount of space required for meta data increases in proportion to the drive's total capacity.

This isn't really a matter that is up for debate. It's been settled for many years now. Every single major Macintosh expert agrees that this is a thing.

Interestingly, not only hasn't this changed with the move to SSD's, the amount of free space that you must maintain on an SSD has actually increased from what it was for RDHD's (not only do SSD's need to have room made available for meta data, but they need some extra space because of the different way that they deal with full or dead blocks of data.)