4 Radnor Corporate Center
Suite 350
Radnor, PA 19087
+1 (888) 507-2270
1000 issues limit
Forums: Troubleshooting and Problems
SergiyThis is a cool feature requested by many users
Hi Stefan,
This is a cool feature requested by many users of Pinnacle 21 Validator. Limitation of 1000 diagnostics records for each issue is configurable. You can manage it.
There are two basic types of data issues. 1. Manual data entry errors 2. Programming or systematic errors. Amount of data entry errors in a study is limited and in most cases it is under 1000 per issue. So, limitation of 1000 diagnostics records per issue does not impact manual data entry errors. A programming error is a different story.
A single programming error may introduce one or more issues affected thousands or even millions of records. In such cases, a validation report with a complete list of diagnostics records may be extremely large. Note that MS Excel cannot open spreadsheets with more than one million (1,048,575) rows. So, while validation report has valid MS Excel structure, it cannot be processed by MS Excel due to large size of report. On the other side, to fix a programming error you do not need to have millions of examples of the same data issue. It is enough to see just few records with a problem to diagnose a programming error. That why we limited a number of records in “Issue Details” tab to 10,000 per issue. Later by multiple requests from our users, we reduced this limit to 1000 records.
If you do not like such approach, you can always modify this default configuration setting.
- In Pinnacle 21 Community, go to “Help” -> “Preferences -> “Validator”
- In “Validator Settings” tab, modify “Excel Message Limit”. For no limit specify 0
For CLI (Command Line Interface), you can use a parameter --report.cutoff. See details at https://www.pharmasug.org/proceedings/2018/DS/PharmaSUG-2018-DS20.pdf
Please remember about MS Excel limitation. Common practice is that when our users need to see all diagnostics records, they generate a validation report in CSV format and then upload it for analysis into SAS instead of MS Excel.
Kind Regards,
Sergiy
Dear Sergiy, thank you for…
Dear Sergiy,
thank you for the in detail explanation. This is very helpful.
BR,
Stefan
Dear P21,
we observed, that P21 reports generated from P21 Community 3.0.2 and 2.2.0 seem to have a limit, to the rows displayed, of 1000 within the details view for a given category of the issue summary.
I have attached two sceenshots to illustrate the problem.
e.g. according to the issue summary, we would expect to find 36657 rows for source=EG and and Pinnacle21 ID = SD1117, however only 1000 rows match the respective filter criteria within the details tab.
Is this a know limitiation of P21 Community ?
Many thanks,
Stefan
Attached Files