PMDA Validation Rules Webinar Q&A
- The PMDA will be requiring Analysis Results Metadata (ARM) for future submissions. Are Pinnacle 21 Community/Enterprise going to support define.xml for analysis results metadata?
Pinnacle 21 Enterprise will add support for ARM at the end of Q1 2016. There are no plans to add ARM support in Community at this time.
- Are there validation checks performed in Enterprise 3.0 that are not done in Community 2.1?
The short answer is yes. However, it depends on what validation rules you compare. When looking at just the FDA and PMDA published rules, both Community and Enterprise have the same checks. The differences are in rules that have been implemented, but not yet published (typically a 9-12 months lag), and rules that are specific to each customer’s tools and processes. There is also a large suite of data composition and quality reports that are only available in Enterprise. These are especially helpful in finding those issues that are not possible to catch with automatic checks.
- When do the new rules come into effect for PMDA submissions?
The rules came into effect on November 18, 2015, the date they were published by PMDA. Companies that are participating in PMDA Pilot or planning electronic data submissions in 2016 should start using the new rules.
- If a study was already validated with Community v2.0 rules in 2015 and submission is planned in 2016, should a pharmaceutical company re-validate the study based on the latest version of validation rules?
Yes, companies should re-validate study data using the latest available validation rules prior to submission. This gives companies a chance to explain newly found issues in Reviewer’s Guide or correct the serious issues (like the PMDA Reject rules).
- What does FDA do with respect to “Error” severity? Do they suspend review?
FDA has not yet published guidance that describes criteria for rejection or suspension of review. They simply ask that companies validate study data prior to submission, correct all issues that can be fixed, and explain the rest in the Reviewer’s Guide. However, with December 2016 quickly approaching things could change.
- In Enterprise, against which version of metadata do you run the checks? Can you have several versions in the “Manage Metadata” tab?
Enterprise validates against CDISC standard metadata and sponsor specified metadata. For example, a company can manage multiple versions of global, therapeutic area, compound, and study-level metadata in Enterprise. These can be defined as extensions of CDISC standards or any custom standard. Once defined and assigned to a particular study, the system will validate and enforce compliance of study data and metadata.
- How is it that these are “standards” when we have 3 different versions of compliance checks with CDISC, FDA, and PMDA rules?
To answer this question, lets first review what it means to be CDISC compliant vs. FDA or PMDA compliant. A common misconception is that preparing data according to business rules and examples in CDISC implementation guides is sufficient for regulatory submission. This of course is not enough. To be FDA or PMDA compliant a sponsor must adhere to multiple standards (CDISC, eCTD, MedDRA, SNOMED, etc.), conform with business rules defined in regulatory guidance (ex: FDA Study Data Technical Conformance Guide), and ensure that data collection process results in high-quality data. This is why there will always be more FDA and PMDA rules than just the ones published by CDISC. The optimal scenario is for FDA and PMDA rules to extend the list published by CDISC focusing on regulatory review requirements and data quality. This is what we promote here at Pinnacle 21 and is the scenario that is supported by both agencies. Now that CDISC is getting closer to publishing SDTM rules, this scenario will finally become possible. And finally, to answer why FDA and PMDA rules are different, just consider that agencies use different review and analysis tools, have different processes, and place different importance (severity) on complying with specific requirements. Actually, both FDA and PMDA have done a great job keeping their requirements consistent especially given the differences just mentioned, which is why their rules differ primarily in severity.
- Why are there IG related rule sets if only the most recent one is critical and will be used by agencies?
The agencies publish rule sets for each supported version of the implementation guides (ex: SDTM-IG 3.1.2, SDTM-IG 3.1.3, etc.). So you need to comply with the most recent rules for the particular version of the implementation guide chosen by your organization.
- Does Community v2.1 validate Medical Device domains?
No, not yet. But it’s on our roadmap for 2016 and we hope to add support for Medical Device studies by the end of the year.
- Where can I find sample CDISC datasets?
Here is a list of publicly available datasets:
- Define-XML v2.0 Release Package – includes sample datasets for ADaM and SDTM 3.1.2
- Metadata Submission Guideline (MSG) Package – includes sample datasets for SDTM 3.1.2
- Sample SEND datasets – made available from Instem
- With what frequency are the validation rules updated?
Pinnacle 21 updates validation rules at least twice per year. FDA and PMDA does not currently provide a schedule for rule releases.
- Is it mandatory to follow CDISC SDTM for all submissions to PMDA and FDA? Does EMA have any timeline for CDISC submission requirements?
FDA requires studies that begin in December 2016 to be submitted electronically in CDISC format. For details refer to our FDA Final Guidance on Study Data Standards webinar. PMDA will begin requiring CDISC submissions in late 2016, but no specific date has been set. Additional information can be found on PMDA website. EMA has not yet provided any guidance or timelines for CDISC submissions.
- Do the check IDs in Pinnacle 21 Community indicate which ADaM validation rules are being implemented? Are there ADaM validation checks defined by Pinnacle that were not part of the checks created by the CDISC ADaM team?
Yes, the Publisher ID column in the Validation Report shows how Pinnacle 21 check IDs map to rule IDs published by CDISC ADaM team. Pinnacle 21 validation rules for ADaM closely follow the rules published by CDISC ADaM team. The only exception is the addition of metadata consistency checks that ensure that ADaM datasets follow the definition in the accompanying define.xml.
- Most data is eventually submitted to both FDA and PMDA. Would you recommend running both FDA and PMDA checks before study finalization?
Yes, we highly recommend that companies adhere to both FDA and PMDA validation rules. And that both validation rule sets are run throughout the study, not just before finalization.
- If I completed my studies and finalized my CSRs, why would I now have to run these through more current checks years later when I file my submission? I cannot imagine the FDA is recommending companies unlock/open numerous completed studies because they have released new validation checks.
This question makes an assumption that validation rules are the sole requirement for submission. Instead validation rules are just a tool to help companies comply with published regulatory guidance, regulations, and best practices. The validation rules improve over time, but can never become “perfect” and cover every possible business rule, regulatory requirement, and data quality check. But that’s no excuse for companies not to follow regulatory requirements or create high-quality data. For example, lets say new rules are introduced to check for acceptable ranges for lab results, and they catch a mapping error that caused your standard results to be off by a factor. Do you fix the data or submit it as is?
- Does the Community version check for the “reject rules” for Define-XML that Crystal Allard shared last week at the PhUSE SDE?
Yes, many of those rules are included in v2.1, but not all. Also, these rules have not yet been officially published by FDA or upgraded to “Reject” severity. So stay tuned for more details. In the meantime, we highly recommend that everyone review Common Errors in Loading SDTM Data to the Clinical Trials Repository presentation.
- How often do you guys release a new Validator version?
Validator will be updated at least twice per year, with more frequent patch releases when necessary.
- PMDA refers to devices according to the slide. If it is a device study but not submitted in Japan, do we use PMDA instead of FDA?
Not sure which slide you are referring to, but neither PMDA nor FDA currently publishes validation rules for medical device studies. However, we have plans to introduce these rules by the end of 2016.
- Do some warnings create a fail-to-load in CTR condition? Or only the errors do?
The fail-to-load conditions for CTR are currently being reviewed based on the analysis performed by FDA CDER OCS team. Once these validation rules are finalized and approved, I expect they will be published using the established FDA process.
- If FDA or PMDA changes requirements to Reject, what advanced notice will we received before companies must comply?
Good question and one that can only be answered by regulatory agencies.
