Error on run of new release Alpha 2

RE: Error on run of new release Alpha 2

Hi Diane, Thank you for posting the error message. When the stream is closed unexpectedly it's an indication that the program's connection with the file was broken prematurely. I've gone ahead and looked at the responsible code, and I can't find an immediate cause in the program. Can you reproduce the error all of the time, or did it happen only once? Also, is the SAS file you're validating on the local computer that you're working on, or on a network? Finally, if I could get the size of the file in bytes that might be useful as well. Regards, Tim

RE: Error happened again, at the same point in the run.

I tried it 2 additional times, and get the same message at the same point in the run. The last record of the SC dataset. Progress says: "Processed 37 records from data source 'SC.xpt'... I don't think I'm losing connection to the network. I can still navigate to it and open the datasets. java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at java.io.FilterInputStream.read(Unknown Source) at org.opencdisc.validator.data.XportSasDataSource.getBuffer(XportSasDataSource.java:434) at org.opencdisc.validator.data.XportSasDataSource.(XportSasDataSource.java:170) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:815) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:794) at org.opencdisc.validator.ValidationEngine.run(ValidationEngine.java:388) at org.opencdisc.validator.ThreadedValidationEngine.run(ThreadedValidationEngine.java:75)

RE: Error happened again, at the same point in the run.

I'm sorry, you're absolutely right. I misread the error message. You're actually not having a problem with the SC.xpt file, but the dataset that's being loaded after that (whichever it is). Is there an empty dataset in the list of files that you're validating? In any case, I'm able to prepare a fix for that, so I'll let you know when that's been uploaded if you'd like to see if that prevents the error. Update - The update has been uploaded. If you download the Validator again, it should be fixed. Regards, Tim

Same error at same point.

Got the another (perhaps the same) error after re-downloading and unzipping the download file. I do have an empty domain, it's ZZ. Do you know which domain should load after SC? In watching the "progress", it runs VS, SUPPAE, RELREC, and SC then crashes. Is there a longer "log" type file I could send you, or is the below message really all that's produced. Let me know what else I can send you to help out. java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at java.io.FilterInputStream.read(Unknown Source) at org.opencdisc.validator.data.XportSasDataSource.getBuffer(XportSasDataSource.java:434) at org.opencdisc.validator.data.XportSasDataSource.(XportSasDataSource.java:170) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:815) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:794) at org.opencdisc.validator.ValidationEngine.run(ValidationEngine.java:388) at org.opencdisc.validator.ThreadedValidationEngine.run(ThreadedValidationEngine.java:75)

Same Java runtime error

Same here. I am using the SDS-ADaM pilot data. It failed at ds.xpt "Processed 254 records from data source 'ds.xpt'..." java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at java.io.FilterInputStream.read(Unknown Source) at org.opencdisc.validator.data.XportSasDataSource.getBuffer(XportSasDataSource.java:436) at org.opencdisc.validator.data.XportSasDataSource.getBuffer(XportSasDataSource.java:422) at org.opencdisc.validator.data.XportSasDataSource.(XportSasDataSource.java:136) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:818) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:797) at org.opencdisc.validator.ValidationEngine.run(ValidationEngine.java:388) at org.opencdisc.validator.ThreadedValidationEngine.run(ThreadedValidationEngine.java:75) P.S. In case the following is helpful. Windows 2000 SP4 java version "1.6.0_05" Java(TM) SE Runtime Environment (build 1.6.0_05-b13) Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing)

RE: Same error at same point.

Hi Diane, It sounds like when you downloaded the Validator again your browser served up a cached version (I had the same thing happen when checking the Alpha 2 release). You can clear your browser's temporary files and try again, or I can upload the validator.jar for you to get individually. The reason I think this is the case is because the potential error-causing line has moved to line 171, but your error message still reports XportSasDataSource.java:170 as being the source. In regards to the log, the file validator.log will contain more information if you go to the lib/log4j/log4j.properties file and change the ERROR to DEBUG. It should be the empty data set causing the issue, based on what the code is doing when it throws that exception, but the order with which the files are gotten isn't well-defined, so I don't know which should come after for certain. Regards, Tim

RE: Same Java runtime error

Hi Tony, Would you mind following the instructions that I gave Diane (below) in regards to the validator.log file? That way we can determine which file it is that's causing the exception to be thrown. Your file is failing because it's getting to the end of the file before it finds information that the SAS specification suggests must be there (a little bit different than the original problem of this thread), so it may be and indication of a larger issue. Regards, Tim

Deleted cookies and temp internet files

I deleted cookies and temporary internet files. Still get the error. Did I not delete the correct stuff? java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at java.io.FilterInputStream.read(Unknown Source) at org.opencdisc.validator.data.XportSasDataSource.getBuffer(XportSasDataSource.java:434) at org.opencdisc.validator.data.XportSasDataSource.(XportSasDataSource.java:170) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:815) at org.opencdisc.validator.ValidationEngine.getSource(ValidationEngine.java:794) at org.opencdisc.validator.ValidationEngine.run(ValidationEngine.java:388) at org.opencdisc.validator.ThreadedValidationEngine.run(ThreadedValidationEngine.java:75) Looking at the VALIDATOR.LOG with the DEBUG option on, it is my empty ZZ dataset that is causing it to die. This is my test dataset for check IR4000 to fire. Diane

RE: Deleted cookies and temp internet files

Hi Diane, You did that and then re-downloaded the Validator, to no success? I'm admittedly not sure why that would be, but you can download this file, unzip it, and then move it into the Validator folder you have now to overwrite the existing version of that component. Then if you try the validation again hopefully we'll see some difference. Alright, that's good to know. The IR4000 check will fire successfully, but unfortunately SAS doesn't explain in their XPORT file specification what the file will look like if it doesn't have any records, so these kinds of quirks prevent the code from reaching that check. That being said, for that reason we're really appreciative of all of this kind of feedback, since creating the various SAS files on our end isn't particularly convenient. Regards, Tim

validator.log

Tim, I followed your suggestion. Apparently, ds.xpt was processed fine. The offending file seems to be define.xml which resides in the same folder, as suggested by these ending entries in validator.log: 223531 [Thread-3] DEBUG org.opencdisc.validator.Main - Ending current validation loop, 254 rows processed in C:\CDISC\Not Shareable\900171_2008_01_22T1920\900171\m5\datasets\CDISCPILOT01\tabulations\ds.xpt, 166811 rows processed total 223531 [Thread-3] DEBUG org.opencdisc.validator.Main - Performing post-processing validations 223531 [Thread-3] DEBUG org.opencdisc.validator.Main - Writing finished processing source message to report instance 223531 [Thread-3] DEBUG org.opencdisc.validator.Main - Finished source processing loop, 11 files processed 223531 [Thread-3] DEBUG org.opencdisc.validator.Main - Attempting to generate a DataSource for C:\CDISC\900171_2008_01_22T1920\900171\m5\datasets\CDISCPILOT01\tabulations\define.xml 223531 [Thread-3] DEBUG org.opencdisc.validator.Main - Attempting to create new XportSasDataSource instance I am using the SDS-ADaM pilot data available from CDISC.org. The folder structure and files are identical to the original source, i.e., compliant to eCTD requirements.

RE:validator.log

Hi Tony, Awesome, that's exactly what I needed to know. The code is meant to check for files that don't appear to be SAS, but there's clearly a loophole. I'm going to go ahead and patch that up today, and I'll let you know when the new version has been uploaded. After that it should be able to process your sources successfully. Thanks for your help! Update The Validator has been updated. Please download the latest version and see if that helps. Regards, Tim

Seems to have worked with the new validator.jar file

Incidentally, the one in that single zip is different than the one I downloaded this morning at 8:55. (2/16 vs 2/20 date stamp) It ran. Thanks! I'll get back to you with any differences in output between our tools (where yours supports a check). Diane

RE: Seems to have worked with the new validator.jar file

Yeah, the most recent versions of both the whole validator and the validaor.zip files should (now that I fixed Tony's problem since you downloaded it as well) be dated 2/24. If it's anything sooner, the file is being cached somewhere and the old version is being sent to you. I'll see if there's any way we can convince the browser or whatever not to do that from our end. In any case, I'm glad to hear that it's working now. Thanks for your patience and sorry for the inconvenience! Regards, Tim

SAS XPORT File Layout

Tim, This will explain how a zero-record XPORT would look: http://support.sas.com/techsup/technote/ts140.html

RE: SAS XPORT File Layout

Tony, We actually used that document to create the SAS XPORT file reader, but in case I accidentally missed it, could you point out where it talks about blank XPORT datasets? The problem that caused the error in Diane's case was that (apparently) the Observation header isn't included in the dataset if there are no Data records, but I didn't see anything that mentioned that was the expected behaviour. Regards, Tim

RE: SAS XPORT File Layout

Tim, If you follow the order of "HEADER RECORD" mentioned in the layout document, the last header would be "HEADER RECORD*******OBS HEADER RECORD!!!!!!!000000000000000000000000000000" (without quotes). A zero-record XPORT file will end right there since there is no need to have anything to delineate end of file. P.S. Because records must stream 80 bytes, each header will automatically have 2 trailing white spaces, i.e., 0x20.

RE: SAS XPORT File Layout

Tony, Yep, you're absolutely right. That record didn't apparently exist in Diane's file though, since the code that was throwing the exception came before the code that expects the OBS header record, which would suggest that the file didn't contain enough data to have had that record present (since we reached the end of the file before all of the expected header records were read). In any case, the Validator should no longer have that problem, so hopefully it won't be an issue either way. Additionally, let me know if the newest version (that should be dated February 24th) helps to solve the exception that you were seeing in regards to the define.xml file. Regards, Tim