EPA Moderated Transaction System (EMTS)
Flow Configuration Document
Version: LI
United States
Environmental Protection
Agency
-------
EPA Moderated Transaction System (EMTS)
Flow Configuration Document
Version: LI
Compliance and Innovative Strategies Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
Prepared for EPA by
PQA, part of
SRA International, Inc.
United States EPA-420-B-10-013C
Environmental Protection . omn
Agency June 2010
-------
of
Table of Contents 1
1 Change Tracking Log 3
1 Change Tracking Log 3
1.1 Version and Component Alignment 4
2 Introduction 5
2.1 How to Use This FCD 6
3 Data Processing 8
3.1 Overview 8
3.2 Implementing the Header Document 8
3.3 Header Document Structure 8
3.3.1 Namespace and Schema 9
3.3.2 Document Header Definitions 9
3.4 EMTS Namespace 12
3.5 Payload 13
3.5.1 Validating the EMTS XML 13
4 Configuring the Network Exchange 14
5 Submission Processing and Feedback 15
5.1 Submission Using a Node or Node Client 15
5.2 Submission from the EMTS Website 18
5.3 Get Submission Status 20
5.4 Retrieve QA Feedback Report 22
5.5 Obtain Transaction History 24
5.6 Obtain Document History 25
5.7 Download Document 26
6 Data Publishing 27
6.1 Overview 27
6.2 EMTS Solicit Flow Steps 28
7 The EMTS Archive Flow 33
7.1 Overview 33
7.2 Archive Submission 33
7.3 EMTS-ARCHIVE Flow 34
7.4 Download Archived Documents 35
8 Web Service Methods 36
8.1 Authenticate 36
8.2 Submit 36
8.3 GetStatus 37
8.4 Query 38
8.5 Solicit 38
8.6 Download 39
Appendix A 1
Node Options 1
Appendix B 1
OTAQ Registration Process 1
B-1 New User Registration 1
B-2 NAAS Node Clients User Registration 1
B-3 NAAS Full Nodes User Registration 2
B-4 NAAS User Revocation 3
Appendix C 1
Procedure for Testing the EMTS Flow 1
Appendix D 1
Accessing EMTS from MyCDXWeb 1
Appendix E 1
Notifications for EMTS Flow 1
-------
EMTS Flow Configuration Document
06/02/2010
Appendix F 1
EMTS Web Service Documents 1
F-1 Intervals for Documents 1
F-2 Document Parameters 1
F-3 Summary of Web Service Documents 2
F-4 Pending Trades 2
F-5 RIN Holding 34
F-6 Transaction History 4§
F-7 Expired Trades 6j£
F-8 Completed Trades 7&
Appendix G 1
EMTS Support 1
-------
EMTS Flow Configuration Document
06/02/2010
October 12, 2009
November 18, 2009
December 23, 2009
January 11, 2010
January 15, 2010
January 18, 2010
March 12,2010
March 31, 2010
1.0
1.0
1.0
1.0
1.0
1.0
1.0.a
1.0.a
Original Draft.
Updates.
Updates describing the
OTAQ Registration process.
Clarification provided in
section 5 on the EMTS data
flow.
Updates for submission and
registration.
Updates.
Updates and addition of
notifications.
Solicit and Archive flow.
Updates to Solicit and
Archive flows. Update to
Notifications and Web
Service Documents.
PQA
PQA
PQA
CGI
CGI
PQA
CGI
PQA
June 2, 2010
1.1
Modification to schema. SRA
Updates to the solicit flow.
Correction to web service
parameter lists. Change in
EMTS support email
address.
-------
EMTS Flow Configuration Document
06/02/2010
1.1 Version and Component Alignment
The following table indicates the version of related documents and components to which this document
applies.
of
FCD V1.1 Solicit method for EMTS dataflow changed from GetEMTSDocuments to
GetEMTSDocument.
Additional information provided to requirement of Organizationldentifier in
Solicit method for GetEMTSDocument request.
Appendix F updated to show new columns added to web service
documents.
EMTS V1.1 PricePerRINDataType and PricePerGallonDataType modified to
Schema minlnclusive value="0".
DET V1.1 PricePerRINDataType and PricePerGallonDataType modified to
minlnclusive value="0".
-------
EMTS Flow Configuration Document
06/02/2010
The purpose of this Flow Configuration Document (FCD) is to describe the operation of the EMTS flow
across the Exchange Network to the CDX Node. This includes data flow between industry users and
EPA that meet the RFS submission requirements.
An FCD is intended to describe the operational aspects of an Exchange Network data exchange. The
specifications included in this document define the supported data services, the approaches, and
processes that are used to exchange information between the EPA and submitters.
This document will describe the data exchange process by which industry users will submit XML files to
the EMTS and retrieve documents from the EMTS.
The EMTS intends to offer outbound services that will allow Exchange Network partners to query data
collected and managed by the EMTS. These outbound data services will continue to be documented
once these capabilities have been designated and developed in a future release.
The scope of this document describes the process of submitting electronically reported EMTS
submissions to EMTS from an EPA Exchange Network 2.0 compliant node, such as the EMTS website,
node, or node client. This document also describes the process for the retrieval of documents available
from the EMTS using the EMTS reports flow.
The high-level process for the EMTS data flow includes the following steps:
1. An EMTS-registered user authenticates to NAAS.
2. The user submits EMTS transaction data to the CDX Node using an Exchange Network 2.0 node
or node client.
3. The user's NAAS credentials are mapped to the user's Web CDX credentials, which are used by
the EMTS for authentication.
4. The CDX Node validates the structure of the XML file.
5. The CDX Node submits the XML file to the EMTS.
6. The EMTS processes the contents of the file.
7. The EMTS notifies the submitter regarding the status of the submission.
8. The user downloads the resulting QA Feedback report.
To obtain documents from the EMTS, the steps are:
1. An EMTS user subscribes to document in the EMTS website.
2. An EMTS-registered user authenticates to NAAS.
3. The user solicits the CDX node for daily/weekly or monthly documents.
-------
EMTS Flow Configuration Document
06/02/2010
4. The user's NAAS credentials are mapped to the Web CDX credentials, which are used by the
EMTS for authentication.
5. The EMTS receives a solicit for documents and either submits these documents to the CDX
Node, or sends a notification that the document was unavailable.
6. The user downloads the documents from the CDX Node.
2.1 How to Use This FCD
This document provides guidance on implementing and using the EMTS flows, an XML-based
submission process through the CDX Node to the EMTS.
The document includes the following main sections:
Submission File Structure
This section describes the various portions that need to be implemented in the EMTS submission file, in
addition to the EMTS XML document. The submission file must adhere to this structure in order to be
processed properly in route through the exchange network to the EMTS. The EMTS flow makes use of
the Exchange Network Header Document to describe the payload content of a Network message. Every
submission to the EMTS must adhere to the Header Document structure.
Configuring the Network Exchange
This section describes how the EMTS Network Exchange is configured for the EMTS data flows.
In addition to this FCD, the submitter should obtain the following documents related to the EMTS Flow.
These documents can be downloaded from http://www.exchangenetwork.net/exchanges/air/emts.htm.
Additional help can be requested from EMTS Support at support@epamts-support.com or contacting the
EMTS Support line at 1-800-385-6164.
EMTS Data Exchange Template
EMTS XSD
EMTS Transaction Instructions
Submission Processing and Feedback
This section of the FCD provides a description of the EMTS flow including submitting files from a node or
node client and submitting files from the EMTS website.
Solicit Flow for Documents
This section of the FCD provides a description of the EMTS flow used for the retrieval of documents.
Archived Files
This section describes how the CDX archives all submitted XML files and the process for users to retrieve
these documents.
-------
EMTS Flow Configuration Document
06/02/2010
Web Service Methods
This section describes the web service methods utilized by the Exchange Network node in support of the
EMTS flow.
Help and Technical Assistance
If you have questions on how to report your transactions to the EMTS or how to use the EMTS website,
technical assistance can be provided by sending your question to support@.epamts-support.com or by
contacting the EMTS Support line at 1-800-385-6164.
-------
EMTS Flow Configuration Document
06/02/2010
3 Data Processing
3.1 Overview
The EMTS submission file consists of a single XML document that can only be submitted from a node,
node client, or the EMTS website. The submission must adhere to the Exchange Header v2.0 document
structure. Information on the header can be found on the Exchange Network website at
http://www.exchangenetwork.net/dev schema/HeaderSpecification v2.0.doc. The Exchange Header xsd
is available at http://www.exchangenetwork.net/schema/header/2/header v2.0.xsd.
The header document provides the following information:
Description of payload; and
Information regarding the submitter and date of submission.
3.2 Implementing the Header Document
The header is required for all Submit operations through the Exchange Network. Every submission file
sent to the EMTS must use the Header Document structure to meet EPA CDX processing requirements.
The header contains information about the submitter and the contents of the payload. The root element
of the header document is the Document element, with two child elements, Header and Payload. The
Payload contains the actual EMTS data, adhering to the structure of the EMTS schema. The figure below
shows the file submission structure.
Figure 1: Submission File Structure
Submission File (.zip)
Document (.xml)
Header (xml)
Payload (EMTS)
Multiple EMTS transactions
v ^
3.3 Header Document Structure
The following table describes the Header Document elements and how they are used for the purposes of
an EMTS submission. Note that most of these elements are not required to be reported in the header,
these data are used as metadata.
-------
EMTS Flow Configuration Document
06/02/2010
3.3.1 Namespace and Schema
The XML namespace for the Exchange Network Document is
http://www.exchangenetwork.net/schema/header/2.
All documents that use the Header 2.0 specification must use this namespace URL. The schema file is
located at:
http://www.exchangenetwork.net/schema/header/2/header v2.0.xsd.
3.3.2 Document Header Definitions
The document header is the first child of the Document tag. The header may contain the elements
defined below:
-------
EMTS Flow Configuration Document
06/02/2010
Table 1: Exchange Header 2.0 Document Elements
of an
ID
hdrXML
Namespace
xsi XML
Namespace
Schema
Location
A unique identifier for the
XML document that is the
payload. This may be a file
name or a unique
identification number that
can be used to identify the
document.
The Exchange Header 2.0
namespace. This is
different from the EMTS
namespace which is
contained in the EMTS
document.
The W3C 2001 XML
schema instance
namespace.
References an XML
Schema document that has
a target namespace.
ID123456789
Yes
xmlns:hdr =
"http://www.exchangene
twork.net/schema/heade
r/2"
Yes
xmlns:xsi =
"http://www.w3.org/2001
/XMLSchema-instance"
xsi:schemaLocation =
"http://www.exchangene
twork.net/schema/heade
r/2
http://www.exchangenet
work.net/schema/header
/2/header v2.0.xsd"
Yes
No
Must be
http://www.exchan
genetwork.net/sch
ema/header/2.
All documents that
use the Header
2.0 specification
must use this
namespace URL.
The first member
of each pair is the
namespace for
which the second
member is the hint
describing where
to find to an
appropriate
schema document.
The following table describes the Header Document elements and how they are used for the purposes of
an EMTS submission.
10
-------
EMTS Flow Configuration Document
06/02/2010
Table 2: Exchange Header 2.0 Elements
AuthorName
Originator of the document.
This should be the name of a
person or a network node ID
if the document is
automatically generated.
Organization The organization to which the
Name author belongs. It may be a
state name, an organization
name, or a company name.
For submissions to the CDX
node, this should be the
name of the organization.
DocumentTitle Title of the document.
John Smith
EMTS
CreationDate This is a timestamp that
Time marks when the document,
including the payload and
header part, was created.
Keywords Description of the payload.
Multiple keywords should be
separated by commas. This
is used only for document
categorization and searching.
Comment Additional comments for
processors.
DataFlowName The name of the data flow
associated with the payload.
2006-04-05T09:30:47-
05:00
EMTS
DataService
Name
Not used by EMTS.
Yes. Tag
must be
present
but null
string can
be
provided.
Yes. Tag
must be
present
but null
string can
be
provided.
Yes. Tag
must be
present
but null
string can
be
provided.
Yes
(Cannot
be
empty).
No.
Used for reference
only.
Used for reference
only.
Reference to the
flow.
Must be in valid
xsd:datetime
format.
No.
No.
No.
(cont.)
11
-------
EMTS Flow Configuration Document
06/02/2010
Table 2: Exchange Header 2.0 Elements (cont.)
SenderContact
Application
Userldentifier
The sender's additional P.O. Box 1234
contact information. It can Richmond, VA
contain sender's electronic
address and/or telephone
numbers where the author
can be reached.
Application specific submitter JohnSmith
identification.
No.
No.
SenderAddress A well-formed URI where
results or reports can be sent.
Property Other properties of the
document using named value
pairs.
Signature An XML signature associated
with the document.
No.
No.
No.
Element is not
used by EMTS. If
data is provided it
will be ignored.
The submitter may
supply his CDX
registered account
information that is
used to access
MyCDX. The CDX
user ID is the
equivalent of the
EMS user
identifier.
Element is not
used by the
EMTS.
Element is not
used by the
EMTS.
Element is not
used by the
EMTS.
3.4 EMTS Namespace
The EMTS namespace uses a URL to reference the location of the current version of the EMTS on the
CDX network, and allows the user to reference it through a prefix "emts:". The major version number of
the EMTS is also included in the namespace. Without this reference, all complex types and XML
elements cannot be validated resulting in an XML file validation error.
Figure 2 provides an XML example which declares the EMTS namespace through the use of the xmlns
(XML Namespace) schema attribute and assigns the namespace a prefix of "emts:".
Figure 2: Declaring the EMTS Namespace
=: e mt s: E MTS x s i: sc h e m a L oc at i o n=" htt p: //www. exc h a ng e n etwo rk. n et/s c h e m a/e mt si 1 E MTS_E MTS_v1.0. xsd"
xmlns:emts="http://www.exchangenetwork.net/schema/emts/1" xmlns:xsi="
ittp://www. w3.org/2001/XMLSchema-instance'1>
12
-------
EMTS Flow Configuration Document
06/02/2010
The declaration of the namespace is included at the top of each XML file, and allows the user to
reference the EMTS complex types and elements with the "emts:" prefix followed by a colon (as in
emts:GenerateTransactionDetail). Each complex type and XML element, including root elements, must
contain the namespace prefix.
The following XML example demonstrates an XML submission file that contains the Exchange Network
Header with an EMTS payload. This file should be zipped before it is submitted to the EMTS.
Figure 3: Header Example
2009-12-18T09 : 22 : 50CindyFerreira
.3:emt3="http: //www.exchangenetwork.net/schema/emts/1"
xmlns:x3i="http://www.wS.org/2001/XMLSchema-instance">
CindyFerreira2009-12-181234567Non-ester Renewable Diesel Retire
RINQuantity>65
3atcrA7oi'jir.e>65
FuelCode>5
Assigr.ir.entCode>l
RINYear>2009
RetireRea3or.Code>70
No comment provided
3.5 Payload
The EMTS supports the inclusion of only one payload per submission file. The CDX node will not route
submissions including more than one payload. However, a payload can contain multiple EMTS
transactions. The format of the payload must be constructed in accordance with the EMTS schema.
3.5.1 Validating the EMTS XML
Prior to transmitting any XML files to EPA, the XML document should be checked for validity. This can be
accomplished using either or both of the following:
EPA's XML schema validation tool. This web-based validation tool is a set of XML web services
for validating XML documents against the associated schemas and custom rules. This can be
found at http://tools.epacdxnode.net/.
A third party XML validation tool (e.g., XML Spy, Liquid XML, etc.) is strongly encouraged. The
validation tool must validate that an XML document is both well-formed and valid against the
EMTS schema specifications.
13
-------
EMTS Flow Configuration Document
06/02/2010
4 the
There two data flows for the EMTS on the Exchange Network which are detailed in the following table.
Table 3: Configuration of Network Exchange Parameters
Flows
Specification
FCD Specifications
Flow Name
Payload Schema
Header Property
Payload Formatting
Structure
Payload File Naming
Convention
GetStatus Responses
Timing
Authentication and User
IDs Mapping
NAAS Authorized User
Accounts
EMTS
Section 5, 6
Submit and Solicit Parameters
securityToken: A security ticket
issued by the service provider or
a trusted service provider.
dataflow: The name of target
dataflow: "EMTS".
Node Submission will support one
payload per document. Only one
document will be provided,
comprised of the payload.
EMTS_EMTS_v1.1.xsd
N/A
The payload must be an XML
document that is compressed as a
ZIP document type.
N/A
All standard GetStatus responses.
See flow specifications in section 5
and 6.
All node users must be registered in
NAAS and their CDX user IDs must
be mapped to their NAAS IDs.
Contact the EMTS Support Desk or
Appendix B for details.
CDX must authorize Submit,
GetStatus, Query, Solicit and
Download to EMTS.
Section 7
Submit Parameters
securityToken: A security ticket
issued by the service provider or
a trusted service provider.
dataflow: The name of target
dataflow: "EMTS-ARCHIVE".
Node Submission will support one
payload per document. Only one
document will be provided,
comprised of the payload.
N/A
N/A
The payload will be an XML
document that is compressed as a
ZIP document type.
N/A
All standard GetStatus responses.
See flow specifications in section 7.
This flow allows access through a
single NAAS account from the EMTS
node to submit XML files to the CDX
node for archiving. This flow is not
used by industry nodes.
CDX must authorize Submit to
EMTS.
14
-------
EMTS Flow Configuration Document
06/02/2010
5 Submission Processing and Feedback
In order to initiate an EMTS flow submission, the user must have a CDX web account to access the
MyCDX website and have registered their organization and facilities through the OTAQ Fuel Registration
process. The user must also have a NAAS account and have permission to participate in the EMTS data
flow. Both node clients and full nodes must use NAAS accounts registered for the EMTS data flow. The
submitter's NAAS ID must be mapped to his CDX Web user ID. See Appendix B: EMTS Registration
Process, for more information on acquiring account and permissions.
5.1 Submission Using a Node or Node Client
The following diagram and steps demonstrate how an EMTS submitter submits an XML file using a node
or node client.
Figure 4: Submission from Node or Node Client
Industry
EMTS
Submitter
Node
CDX (NAAS, CDX Node, CDX Web)
QA Server
12. Return XML
Validation
Report
11. Validate
-3. Submit
-8. Response
10. Check user
Registration -
And Approval
4-
6. Archive
- 7. Generate
Transaction id
CCXNode
EMTS
Data Flow
4. Validate
OTAQ Registration
Database
Validate Payload
" "^
*-
17
1
21
QAFec
.17. Response-
19. Notify-
20. Submit
13. Create token
9. Map NAAS user
To CDX account
NAAS
. Authenticate
CDX (EMTS)
-18. Process
-16. Queue File
EMTS Node
15. Validate
1. The submitter logs onto their node client or full node application. Depending on the node
application, this step may or may not be required.
2. The submitter logs in to NAAS via the CDX Node using his NAAS user ID. The industry node
issues the authentication web service to the CDX Node. The following end point URL are used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
15
-------
EMTS Flow Configuration Document
06/02/2010
The industry node must pass the following parameters:
userld: NAAS user ID
credential: Password
domain: '"default"
authenticationMethod: "password"
On success, the CDX Node will return a security token to the industry node. Upon failure, the
CDX Node will return an Authentication SOAP Fault.
The following sequence is performed as a synchronous set of actions. If an action fails, the CDX
Node returns a failed submission response containing error information and the transaction ends.
3. The industry node submits an XML file to the CDX Node. The following end point URLs are used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The industry node must pass the following parameters:
security token: Security token received in step 2.
transactionld: Not used.
- dataflow: "EMTS"
flowOperation: Not used.
recipient: Not used.
documents: One document containing the zipped EMTS XML document.
4. The CDX Node validates the security token against NAAS. This will authenticate the submitter
and authorize the submission to the EMTS.
5. The CDX Node validates the payload. The submission payload can contain only one document
which must be compressed and be a ZIP document type.
6. The CDX Node archives the submitted document.
7. The CDX Node generates a unique transaction ID and assigns it the status of "RECEIVED". The
submission CDX Transaction ID uniquely identifies the submission, and can be used to check the
status of the submission.
8. If previous operations were successful the CDX Node returns a Submit response to the industry
node:
transactionld: The unique CDX Transaction ID for the submission.
- status: RECEIVED
statusDetails: Your submission has been accepted and can be tracked using the CDX
transaction id. The submitter node is expected to store the returned CDX transaction ID for
further processing.
If the previous operation fails, the CDX Node returns a SOAP Fault to the industry node and
archives the returned transaction for further processing. See section 5.3, "Get Submission
Status", for more details on the submit web service.
The following sequence is performed asynchronously after the submit response is sent back to the
submitting node.
9. The CDX Node updates the status to "PROCESSING". The CDX Node maps the submitter's
NAAS credentials to his Web CDX credentials. The submitter's Web CDX credentials will be
16
-------
EMTS Flow Configuration Document
06/02/2010
submitted to the EMTS and used by the EMTS for submission authentication and authorization. If
validation fails, the CDX Node will send an email to the submitter and the CDX Node
Administrator regarding the failure.
10. The CDX Node verifies that the user is registered and active in the OTAQ Registration database.
If the user is not active or not registered in the OTAQ Registration database, the transaction
status is set to "FAILED" and the transaction is ended. The CDX Node will send an email to the
submitter and the CDX Node Administrator regarding the failure.
11. The CDX Node validates the XML file against the QA Server, which is performed asynchronously.
After completion, the QA Server submits an XML Validation report to the CDX Node. The CDX
Node archives the XML Validation report. This XML validation report can be downloaded by the
submitter node using the download web service. See section 5.7, "Download Documenf, for
more details on the download web service.
12. If the XML document does not pass XML validation, then the transaction status is set to "FAILED"
and the transaction is ended. The CDX Node sends an email to the submitter regarding the
failure. The email text contains a list of XML validation errors.
13. The CDX Node creates a security token that includes the submitter's Web CDX user ID, email
address, and affiliation. This is mapped from the submitter's NAAS user ID in step 9 above.
14. The CDX Node submits the XML file to the EMTS node, using the security token created in step
13. The CDX Node will resubmit three times if the submission to the EMTS node is not
successful. If the CDX Node successfully submits the document to the EMTS Node, the
transaction status is set to "PENDING". If the submission fails, the transaction status is set to
"FAILED" and the transaction is ended. The CDX Node sends email to the submitter and the
CDX Node administrator regarding the failure.
15. The EMTS Node validates the security token with NAAS and extracts the submitter's Web CDX
user ID from the security token.
16. The EMTS Node queues the submission for processing.
17. If steps 15-16 were successful, the EMTS Node returns the CDX transaction ID to the CDX
Node. If processing fails, the EMTS Node returns a SOAP fault to the CDX Node. The CDX
Node sets the transaction status to "FAILED" and the transaction is ended. The CDX Node
sends an email to the submitter and the CDX Node administrator regarding the failure.
The following sequence is performed asynchronously after the submit response action in step 17.
18. The EMTS processes the submission.
19. If processing is successful, the following steps are performed:
A Notify request is sent to the CDX Node providing the final status of the submission.
The CDX Node ends the transaction and updates the transaction status to "COMPLETED".
The CDX Node sends an email to the submitter regarding the completion of transaction.
17
-------
EMTS Flow Configuration Document
06/02/2010
20. If processing fails, the following steps will be performed:
The EMTS Node submits the QA Feedback report to the CDX Node. The document contains
a zipped QA Feedback report. See section 5.4, "Retrieve QA Feedback Report", for
additional information regarding retrieving the report.
The CDX Node archives the QA Feedback report.
The EMTS Node issues a Notify request to inform the CDX Node of the failure.
The CDX Node ends the transaction and updates Transaction status to "FAILED".
The CDX Node sends an email to the submitter regarding the failure of a transaction.
5.2 Submission from the EMTS Website
The following diagram and steps demonstrate how an EMTS submitter submits an XML file from the
EMTS Website. The EMTS Website has a node client interface customized for the EMTS flow.
Figure 5: Submission from EMTS Website
Industry
EMTS 1. Login
Submitters
CDX (CDX Web, CDX Node, NAAS)
2. Click on EMTS link-
MyCDXWeb
QA Sever
15. Return XML
- Validation
Report
-3. Return error if unregistered user-
10. Archive
11. Generate
Transaction id
9. Store Information
r
Validate Pa rload
14. Validate
13. Check user
Registration
And Approval
4-
u
CDX (EMTS)
n r-4. Select XML file
EMTSWeb
17. Submit-
). Response
!. Notify
19. Queue File
r
12. Response
23. Submit
QA Feedback Report"
7. Validate-
OTAQ Registration
Database
16. Create token
NAAS
-18. Validate-
21. Process
I
5. Create Token
1. The user logs in to the MyCDX Website.
2. The user clicks the OTAQEMTS:OTAQ EMTS link. The user's CDX credentials are passed to the
EMTS website for auto sign-on to the EMTS.
3. If the user does not have permission to operate in the EMTS website, the user is given an error
message and redirected back to the MyCDX website.
18
-------
EMTS Flow Configuration Document
06/02/2010
The following sequence is performed as a synchronous set of actions:
4. The user selects an XML file to submit. The File Submission interface is customized for the
EMTS data flow. The CDX Node URL, data flow name, and other submission parameters are
provided automatically by the EMTS. The EMTS verifies that the document is zipped, and that
the XML conforms to the Exchange Network Header 2.0 and EMTS xsd specifications.
5. The EMTS creates a security token that contains the user's CDX Web user ID, email address,
and affiliation.
6. The EMTS submits the XML file to the CDX Node, using the security token created in step 5.
7. The CDX Node validates security token against NAAS which will authenticate the submitter and
authorize the submission to the EMTS.
8. The CDX Node validates the payload. The submission payload can contain only one document.
The document must be compressed and be a ZIP document type.
9. The CDX extracts the submitter's Web CDX user ID, email address, and affiliation and archives
this information for future use.
10. The CDX Node archives the submitted document.
11. The CDX Node generates a unique transaction ID and updates the status to "RECEIVED". The
submission CDX Transaction ID uniquely identifies the submission, and can be used to check the
status of the submission.
12. If previous operations are successful, the CDX Node returns the following Submit response to the
EMTS:
transactionld: The unique CDX Transaction ID.
- status: RECEIVED
statusDetails: Your submission has been accepted and can be tracked using the CDX
transaction id.
If the previous operation fails, the CDX Node returns a SOAP fault to the submitter node. The
submitter node archives the returned transaction for further processing. See section 5.3, "Get
Submission Status", for additional information regarding the submission status.
The following sequence is performed asynchronously after the submit response is sent back to the
EMTS.
13. The CDX Node updates the status to "PROCESSING", and verifies that the user is registered and
active in the OTAQ Registration database. If the user is not active or not registered in the OTAQ
Registration database, the transaction status is set to "FAILED" and the transaction is ended.
The CDX Node sends email to the submitter and the CDX Node Administrator regarding the
failure.
14. The CDX Node performs XML validation against the QA Server. The QA Server performs XML
validation asynchronously. After completion, the QA Server submits an XML validation report to
the CDX Node. The CDX Node archives the XML validation report.
15. If the document did not pass XML Validation, the transaction status is set to "FAILED" and the
transaction is ended. The CDX Node will send an email to the submitter regarding the failure.
XML validation content is provided within the email.
19
-------
EMTS Flow Configuration Document
06/02/2010
16. The CDX Node creates a security token that includes the submitter's Web CDX user ID, email
address, and affiliation. This is extracted from the security token in step 8 above.
17. The CDX Node submits the XML file to the EMTS Node, using the security token created in step
16. The CDX Node will retry three times if submission to the EMTS Node is not successful. If the
CDX Node successfully submits the document to the EMTS Node, the transaction status is set to
"PENDING". If the submission fails, the transaction status is set to "FAILED" and the transaction
is ended. The CDX Node sends email to the submitter and the CDX Node Administrator
regarding the failure.
18. The EMTS Node validates the security token with NAAS and extracts the submitter's Web CDX
user ID from security token.
19. The EMTS Node queues the submission for processing.
20. If steps 18-19 were successful, the EMTS Node returns the CDX transaction ID to the CDX
Node. If failed, the EMTS Node returns a SOAP fault to the CDX Node. The CDX Node sets the
transaction status to "FAILED" and the transaction is ended.
The following sequence is performed asynchronously after the submit response action in step 19.
21. The EMTS processes the submission.
22. If processing was successful, the following steps are performed:
A Notify request is sent to the CDX Node providing the final status of the submission.
The CDX Node ends the transaction and updates the transaction status to "Completed".
The CDX Node sends an email to the submitter regarding the completion of transaction.
23. If processing fails, the following steps will be performed:
The EMTS Node submits the QA Feedback report to the CDX Node. The document contains a
zipped QA Feedback report. See section 5.4, "Retrieve QA Feedback Report', for additional
information regarding retrieving the report.
The CDX Node archives the QA Feedback report.
The EMTS Node issues a Notify request to inform the CDX Node of the failure.
The CDX Node ends the transaction and updates Transaction status to "FAILED".
The CDX Node sends an email to the submitter regarding the failure of a transaction.
5.3 Get Submission Status
The following steps demonstrate how the industry node obtains the current transaction status. This
functionality is applicable to all node and node clients.
1. The submitter logs into NAAS using his NAAS user ID. NAAS returns a security token to the
submitter node.
2. The industry node issues the GetStatus request using one of the following end point URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
20
-------
EMTS Flow Configuration Document
06/02/2010
The following parameters are used:
securityToken: Security token returned in step 1.
transactionld: Transaction ID returned in section 5.1, step 8.
3. The CDX Node validates the security token against NAAS.
4. If the user was authenticated and authorized, the CDX Node returns a StatusResponse
containing status information:
transactionld: Passed Transaction ID.
status: CDX Node returns any of the following statuses:
o RECEIVED: Submission was received and steps 4-8 from section 5.1 were successfully
performed.
o PROCESSING: Submission was received and steps 9-13 were successfully performed.
o PENDING: Submission was successfully distributed to EMTS Node.
o COMPLETED: Submission was received by EMTS Node and processed.
o FAILED: Submission failed.
statusDetail: A value that describes the failure if the CDX Node returns FAILED as
transaction status.
If the transaction status is "FAILED" then the submitter node can extract the following status
details from the GetStatus response.
Table 4: Status Detail Codes and Descriptions
1 E_XmlValidation An XML schema validation failure occurred on the EMTS
document.
2 E_ApplicationldMapping CDX is unable to find a CDX Web user identification based on
the NAAS identification.
3 EjDtaqRegNotRegistered The CDX was unable to verify that the submitter is a registered
user in the OTAQ Registration database.
4 EjDtaqRegNotApproved The CDX was unable to verify that the submitter is approved in
the OTAQ Registration database.
5 E_Distribution The CDX Node was unable to distribute the document to the
EMTS.
6 E_BackendProcessing The Document failed processing in the EMTS.
21
-------
EMTS Flow Configuration Document
06/02/2010
5.4 Retrieve QA Feedback Report
The EMTS will submit a QA Feedback report to the CDX Node upon a failed submission file. A QA
Feedback report is not generated if the submission was successfully processed. The report contains a
complete list of errors encountered when processing a submitted file. The following steps demonstrate
how the industry node obtains the QA Feedback report using web services methods.
Figure 6: Retrieve QA Feedback Report
Industry
Node
CDX
\. Get Status
-5. Response-
6. Query For Document
-7. Return Document Identifier.
9. Return Document-
2. Submit QA Feedback Report
3. Archive
Report
4-
Node Database
EPA
1. Generate Report
1. When a file fails due to business content checks, the EMTS automatically generates a QA
feedback report. Reports are generated on ten minute intervals.
2. The EMTS Node submits the QA Feedback Report to the CDX Node.
3. The CDX Node archives the report.
4. The industry node issues a GetStatus request using one of the following end point URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
5. The industry node receives a response.
6. If the industry node receives the status details value of "E_BackendProcessing" as a result of the
GetStatus web service method, the industry node invokes the Query web service to obtain the
document ID for the QA Feedback report.
22
-------
EMTS Flow Configuration Document
06/02/2010
The following parameters are used:
securityToken: Security token.
- dataflow: NODE
request: GetDocuments
parameters:
o TransactionldParam: Submission CDX transaction ID.
o Category: REPORT
Note that the dataflow for this service is NODE and not EMTS. This service may not return the same
responses or operate the same using any other dataflow name.
7. The CDX Node returns a document history XML file to the industry node. This XML will contain
transaction and document information, as shown below. The industry node uses the document
ID contained in the document history file to retrieve the document from the CDX Node.
transction id
document iddocument nameREPORTFAILEDXML/ZIP yyyy-mm-ddThh:mm:cc document sizeproperty nameproperty value
8. The industry node invokes the Download web service to the CDX Node.
The following parameters are used:
securityToken: A security token received in authentication step.
- dataflow: EMTS
transactionld: The submission CDX transaction ID.
documents: User the following parameters:
o documentName: Empty string.
o documentFormat: Empty string.
o documentContent: Empty string.
o documentld: document ID retrieved in step 5 above. When the documents parameter is
empty, the CDX Node will return all documents associated with the Transaction ID.
When a document identifier is specified as the document's parameter, the node will
return only the associated document.
23
-------
EMTS Flow Configuration Document
06/02/2010
9. The CDX Node retrieves the QA Feedback report from the archive using the CDX transaction ID
and document ID and returns the QA Feedback report to the industry node.
The industry node receives and stores the QA Feedback report.
5.5 Obtain Transaction History
The following steps demonstrate how the submitter node can obtain the transaction history using the
Query web service. This functionality is applicable to node and node clients.
1. The submitter logs into NAAS using his NAAS user ID. NAAS returns a security token.
2. The industry node invokes the Query web service method using one of the following end point
URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The following parameters are used:
securityToken: Security token returned in step 1.
- dataflow: NODE
request: GetTransactionHistoryByTransactionld
parameters: TransactionldParam (Transaction ID returned by CDX Node in step 9 section
5.1).
3. The CDX Node validates the submitter node user via NAAS.
4. The CDX Node returns a SOAP fault to the industry node if validation fails.
5. The CDX Node returns a transaction history XML to the industry node. This XML contains the
transaction and document information as shown below:
transaction xmlns="">
transaction id Received/Processing/Pending/Pending/Completed/Failed EMTSyyyy-mm-ddThh:mm:ccdocument idfile namefile typedocument status
6. The industry node archives the transaction history.
24
-------
EMTS Flow Configuration Document
06/02/2010
5.6 Obtain Document History
The following steps demonstrate how the industry node obtains the document history for a transaction
using the Query web service method. This functionality is applicable to node and node clients.
1. The submitter logs into NAAS using his NAAS user ID. NAAS returns a security token.
2. The industry node invokes the Query web service method using one of the following end point
URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The following parameters are used:
securityToken: Security token returned in step 1.
- dataflow: NODE
request: GetDocuments
parameters:
o TransactionldParam: Transaction ID returned by the CDX Node in step 9 section
Category: Document category (SUBMISSION, VALIDATION_REPORT, REPORT).
When the category parameter is null the node will return all associated documents.
3. The CDX Node validates the industry node user via NAAS.
4. The CDX Node returns a SOAP fault to the industry node if validation fails.
5. The CDX Node returns the document history for a submission. The XML contains the
Transaction ID and document information as shown below:
transction id
document iddocument nameSUBMISSION/VALIDATION_REPORT/REPORTPENDING/PROCESSED/COMPLETED/FAILEDXML/ZIP yyyy-mm-ddThh:mm:cc document sizeproperty nameproperty value
25
-------
EMTS Flow Configuration Document
06/02/2010
5.7 Download Document
The following steps demonstrate how the industry node downloads a document using the Download web
service method. This functionality is applicable to node and node clients.
1. The submitter logs into NAAS using his NAAS user ID. NAAS returns a security.
2. The industry node invokes the Query web service method using one of the following end point
URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The following parameters are used:
securityToken: The security token received in step 1.
- dataflow: EMTS
transactionld: The CDX transaction ID for the submission. The transaction ID must be the
same Transaction ID issued by the CDX Node (see step 8, section 5.1).
documents: An array of NodeDocumentType. When the documents parameter is empty, the
CDX Node will return all documents associated with the Transaction ID. When a document
identifier is specified as the document's parameter, the node will return only the associated
document.
3. The CDX Node authenticates and authorizes the industry user via NAAS.
4. The CDX Node returns an Authentication/Authorization SOAP fault to the industry node.
5. The CDX Node shall return the requested document.
Note that the dataflow for this service is EMTS. If you use the NODE dataflow for this service, you will
receive a different response status and message.
26
-------
EMTS Flow Configuration Document
06/02/2010
6.1 Overview
The CDX Node supports the capability for industry nodes to request and retrieve EMTS documents. The
CDX Node implements the EMTS Solicit service as a part of the EMTS flow which is available to all
industry nodes.
Industry users must register for the types of documents they wish to retrieve by utilizing the subscription
functionality on the EMTS website. Subscription services should be reviewed periodically and updated to
reflect the industry users need for retrievable documents. The industry user can select from several
reports, each of which is described in Appendix F, "EMTS Web Service Documents". For each
document, the user must identify the time interval and format requested.
During the EMTS testing period, all documents will only be available in CSV format and are generated
on either a daily or weekly basis. More documents, formats, and time intervals will be available in future
releases. This FCD will be updated periodically as additional documents and options become available.
The following section describes the process for subscribing to and retrieving documents from the EMTS.
Figure 7: Solicit Reports from the EMTS
Industry
Industry
Node
13. GetStatus
-14. Response
15. Download
-16. Response-
CDX
3. Archive
Solicit
request
0. Update Status
Node database
EPA
6. Validate and
6. Solicit
7. Response-
9. Notify
-H.Submit-
1. Generate Report
EMTS Node
8. Query for
Document
EMTS Database
27
-------
EMTS Flow Configuration Document
06/02/2010
The EMTS will run scheduled jobs nightly, generating documents as requested by subscribers.
Documents are available for download early the following day. The industry node invokes the Solicit web
service method to the CDX Node using the procedures below. The CDX Node will archive the solicit
request for asynchronous processing. The CDX Node will then send a Solicit request to the EMTS Node
containing the document request information. The EMTS Node will log the Solicit request then retrieve
the document that was created during the overnight scheduled job. This document is then submitted to
the CDX Node. The CDX Node will archive the report and wait for a GetStatus request from the industry
node. In the response, the document identifier information is provided and the industry user issues a
Download request. These documents may also be retrieved manually from the EMTS website should the
user not wish to use the Solicit flow services.
6.2 EMTS Solicit Flow Steps
1. The EMTS creates documents in accordance with an industry user's subscription request. These
documents are generated on a regularly scheduled basis and are available for subscribers the
following morning. The name of the document, date of creation, and the document contents are
stored in the EMTS document log.
2. To determine if subscription documents are available on the EMTS Node, the industry node
invokes the Solicit web service method using following end point URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The following parameters are used:
securityToken: The security token received from the Authentication response.
- dataflow: EMTS
request: GetEMTSDocument
recipient: Leave empty.
notificationURI: Leave empty.
parameters:
o documentName: Name of one of the web service documents (see Appendix F, "EMTS
Web Service Documents"). This is a mandatory string.
o organizationldentifier: Conditionally required string parameter.
o documentDate: Leave empty.
When specifying an optional parameter, you may pass the empty parameter in one of two ways;
no key and no value, or with the key and an empty string.
3. The CDX Node validates the Solicit request ensuring it contains valid and required parameters.
The CDX Node archives the Solicit request.
28
-------
EMTS Flow Configuration Document
06/02/2010
4. The CDX Node generates a CDX Transaction ID associated with the Solicit request. The status
to set to "RECEIVED". The CDX Node returns a StatusResponse to the industry node containing
the following information:
transactionld: CDX Solicit Transaction ID.
status: "RECEIVED" if successful, "FAILED" if error occurs.
statusDetail: Empty.
The industry node stores the returned CDX Transaction ID for reference.
The following sequence is performed asynchronously after the solicit response action in step 4.
5. The CDX Node issues a Solicit request to the EMTS Node. As part of this solicit request, the
CDX Node maps the NAAS user ID to the CDX Web user. If the mapping is successful, the CDX
Node creates a security token that includes the submitter's Web CDX user ID, email address, and
affiliation. The CDX Node issues the Solicit service using the following endpoints:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/Dataflow2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/Dataflow2Service
The following parameters are used:
securityToken: Received in 2.1.
- dataflow: EMTS
request: GetEMTSDocument
recipient: CDX Node endpoint for report submission.
notificationURI: Empty.
parameters:
o documentName: Name of one of the web service documents (see Appendix F, "EMTS
Web Service Documents"). A document name is required.
o organizationldentifier: Conditionally required string parameter.
o documentDate: Leave empty.
6. The EMTS Node validates the CDX Node and extracts the CDX Web user ID from the security
token and logs the request.
7. The EMTS Node generates a new CDX transaction ID and returns it with the response.
8. The EMTS queries the document log for the associated document. The EMTS Node either
submits the document to the CDX Node, or sends a Notify of failure.
The following sequence is performed if the solicit results in failure.
9. If the EMTS Node sends a NOTIFY of "FAILURE", this indicates the EMTS did not have a
document that matched the parameters supplied in the Solicit request. The EMTS Notify
response contains the following information sent to the CDX Node.
The following parameters are used:
SecurityToken: Security token will be received from regular authenticate against CDX Node.
NodeAddress: Empty.
29
-------
EMTS Flow Configuration Document
06/02/2010
- dataflow: EMTS
messages (An array of notification messages consisting of one message).
objectld: EMTS Solicit transaction ID obtained in step 7.
messageCategory: Status
messageName: "Status Notification"
- status: "FAILED"
statusDetail: EMTS processing failure reason as shown in the table below.
Table 7: Notify Failure Processing Reason
The user does not exist.
The user is not affiliated with the organization.
The organization does not exist.
The document name does not exist.
The organization has not subscribed to the document.
The document does not exist.
10. The CDX Node extracts the EMTS Solicit transaction ID from the set of passed parameters and
retrieves the CDX Node Solicit transaction. The CDX Node updates the status details of the
transaction.
The following sequence is performed if the solicit results in success.
11. The EMTS submits the requested document to the CDX Node using the following end point
URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/Dataflow2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/Dataflow2Service
The EMTS Node passes the following parameters:
securityToken: Security token received in step 2.
transactionld: Solicit transaction ID returned to CDX Node in step 2.9.
- dataflow: "EMTS"
flowOperation: "solicitResponse"
recipient: Leave empty.
notification: Leave empty.
documents: One document containing a zipped document.
12. The CDX Node receives the document and extracts the EMTS solicit transaction ID from the
parameter list. The CDX Node associates the transaction related to the EMTS Solicit transaction
and sets the CDX transaction status to "COMPLETED".
30
-------
EMTS Flow Configuration Document
06/02/2010
The following sequence is performed at the CDX Node after the response from the industry node Solicit
request.
13. The industry node issues a GetStatus request to the CDX Node using the following endpoint
URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The following parameters are used:
securityToken: Security token returned during authentication.
transactionld: CDX Solicit transaction ID.
14. The CDX Node validates the user via NAAS. If the user is validated, the CDX Node returns a
StatusResponse containing the status information using the following parameters:
transactionld: Passed transaction ID.
status: One of the following statuses:
o RECEIVED: Solicit request was successfully received, however the request to EMTS
Node has not been sent yet.
o PENDINGThe solicit request was successfully sent to the EMTS.
o PROCESSING: The EMTS Node received the request from the CDX Node and is
currently processing the request.
o COMPLETED: The EMTS Node has processed the request and the document is
available for download.
o FAILED: The process failed.
statusDetails: One of the errors messages shown below.
Number
1 E_ApplicationldMapping
2 E_Solicit
3 E_User
4 E_Affiliation
5 EjDrganization
6 E_DocumentName
7 EjDrganizationDocument
8 E DocumentNotAvailable
Description
CDX is Unable to Find CDX Web User Identification based on
NAAS Identification.
CDX Node Failure to perform Solicit to EMTS.
The user does not exist.
The user is not affiliated with the organization.
The organization does not exist.
The document name does not exist.
The organization has not subscribed to the document.
The document does not exist.
31
-------
EMTS Flow Configuration Document
06/02/2010
15. If the transaction status is "COMPLETED", the industry node invokes the Download web service
method using following end point URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The following parameters are used:
securityToken: A security token.
- dataflow: "EMTS"
transactionld: The Solicit transaction ID. It must be the same transaction ID issued by the
CDX node as response to the Solicit request.
documents: Empty.
16. The CDX Node returns the document to the industry node. The document returned will be either
the requested document or the resulting error report containing the reason for failure.
32
-------
EMTS Flow Configuration Document
06/02/2010
7 The EMTS Archive Flow
7.1 Overview
The EMTS submits XML documents to the CDX Node containing transactions performed by the industry
users from the EMTS website. The EMTS Node submits these documents on a weekly basis to the CDX
Node for archiving. Industry nodes can retrieve the transaction history, using the Query web service and
download documents using the Download web service. The EMTS will pass the user credentials (CDX
Web User ID, email, and affiliation) as a part of security token. The CDX Node will archive the
submission together with the user credentials. Industry Nodes can retrieve transaction history using the
Query web service and download documents using the Download web service. The CDX Node will
authenticate and authorize document submission and retrieval.
The flow described in section 7.2 is not utilized by industry nodes directly. Industry users that wish to
retrieve archived documents from the CDX Node would use the query, and download services described
in section 7.3
7.2 Archive Submission
1. The EMTS Node issues the CreateSecurityToken web service using following endpoint URL:
Test: https://naas.epacdxnode.net/xml/securitvToken vSO.wsdl
Prod: https://cdxnodenaas.epa.gov/xml/securitvToken vSO.wsdl
The following parameters are passed:
trustee: Admin user ID.
credential: EMTS node admin user password.
domain: default
token: csm
- issuer: CDX-Web RSO
authMethod: password
subject: EMTS node operator NAAS ID.
subjectData: URL encoded key value pair consisting of:
o CDX Web user ID;
o User email; and
o User affiliation.
IP: IP address of the EMTS node.
2. The EMTS submits a zipped document to the CDX Node using the following end point URL:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/WeblntegrationService
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/WeblntegrationService,
The following parameters are used:
securityToken: Security token received in step 1.
33
-------
EMTS Flow Configuration Document
06/02/2010
Transactionld: Not used.
- Dataflow: EMTS-ARCHIVE
FlowOperation: Empty.
Recipient: Not used.
Documents: One document containing zipped EMTS XML document.
3. The CDX Node validates the security token via NAAS. This authenticates and authorizes EMTS
to archive.
4. The CDX Node validates the payload. The submission payload can contain only one document
which must be compressed and be a ZIP document type.
5. The CDX Node archives the submitted document.
6. The CDX Node generates a CDX Transaction ID and assigns it the status of "COMPLETED".
The submission CDX Transaction ID identifies the submission, and can be used later by EMTS to
check the status of the submission, obtain transaction history, and download the document.
7. The CDX Node archives the CDX Web user ID, email, and affiliation in association with the CDX
Transaction ID.
8. If successful, the CDX Node returns the following Submit response to the EMTS Node:
transactionld: The CDX Transaction ID.
- status: COMPLETED
statusDetails: Your submission has been accepted and can be tracked using the CDX
transaction id. If failed, the CDX Node returns a SOAP Fault to the EMTS Node.
9. EMTS Node archives returned transaction for further processing.
7.3 EMTS-ARCHIVE Flow
The following steps demonstrate how the industry node obtains archived files for a specified user using
the Query web service method.
1. Industry Node invokes the Query web service method using following end point URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
The following parameters are used:
security Token: The security token received in authentication.
- dataflow: NODE
request: GetTransactions
parameters:
o dataflow: "EMTS-ARCHIVE"
o status: COMPLETED
o startDate: The start date requested (format: "yyyy-mm-dd"). If a date is not specified, all
transactions are returned.
o endDate: The end date requested (format: "yyyy-mm-dd"). If a date is not specified, all
transactions are returned.
o user: CDX Web user ID.
34
-------
EMTS Flow Configuration Document
06/02/2010
2. The CDX Node validates the Industry Node user via NAAS.
3. The CDX Node returns a SOAP Fault to the Industry Node if validation fails.
4. The CDX Node returns a transaction history XML to the Industry Node in the following format:
transaction xmlns="">
_8208d68b-972f-4183-af35-82b3224deceb2009-12-17T10:10:58COMPLETEDEMTS-ARCHIVE
transaction xmlns="">
_91a30409-6c4c-4ead-8bOf-79d4c80c92742009-12-16T21:10:35FAILED/COMPLETED EMTS-ARCHIVE _79b2fOa9-ba83-4cd2-84ad-27d7389104f62009-12-04T11:45:38COMPLETEDEMTS-ARCHIVE
7.4 Download Archived Documents
The following steps demonstrate how the industry node downloads archived documents using the
download web service method.
1. The industry node invokes the Download web service method using following end point URLs:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/Dataflow2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/Dataflow2Service
The following parameters are used:
- securityToken: A security token received in authentication.
- dataflow: EMTS-ARCHIVE
- transactionld: A transaction ID for the submission. It is transaction ID returned in step 4 of
Obtain EMTS-ARCHIVE Transaction section.
- documents: Documents parameter is empty.
2. The CDX Node validates the user via NAAS.
3. The CDX Node returns a SOAP fault to industry node if validation fails.
4. The CDX Node returns the requested document to the industry node.
35
-------
EMTS Flow Configuration Document
06/02/2010
8 Web Service Methods
The following section describes the web service methods utilized by an Exchange Network 2.0 node to
participate in the EMTS flow.
Please see the Exchange Network 2.0 functional specifications for detailed information of Web services.
WSDL's for these web service methods can be obtained at:
http://exchangenetwork.net/schema/node/2/node v20.wsdl
8.1 Authenticate
The Authenticate method is used to obtain a security token from the CDX Node. This token is passed in
any subsequent method calls in the exchange between the submitting node and the CDX Node.
The following URLs may be used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
Authenticate Parameters:
userld: The submitter's NAAS user ID.
credential: The submitter's NAAS password
domain: default
authenticationMethod: password
Returns:
securityToken: A security token used to identify the submitter.
8.2 Submit
The Submit method is used to submit an XML file to EMTS.
The following URLs may be used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
Submit Parameters:
securityToken: The security token issued by NAAS and returned from the Authenticate method.
This security token can be used for several submissions as long as the token has not expired.
transactionld: Leave empty.
dataflow: EMTS
flowOperation: Leave empty.
recipient: Leave empty.
notificationURI: Leave empty.
36
-------
EMTS Flow Configuration Document
06/02/2010
Documents: An array of type nodeDocument. Each nodeDocument structure contains a single
submission file. The EMTS requires a single nodeDocument.
Returns:
A Submit response with the following data:
transactionld: A unique CDX Transaction ID.
status: Text describing the submission's status ("Received" or "Failed").
statusDetails: Additional details regarding the submission.
8.3 GetStatus
The GetStatus method is used to obtain the transaction status of a submission.
The following URLs may be used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
GetStatus Parameters:
securityToken: The security token issued by NAAS and returned from the Authenticate method.
transactionld: The Transaction ID issued by the CDX Node from the Submit or Solicit service.
Returns:
A GetStatus response with the following data:
statusResponseType containing following information:
o transactionld: The Transaction ID issued by the CDX Node from the Submit service.
o status: Text describing the submission's status ("Received", or "Failed").
o statusDetail: Additional details on the Status description shown in the table below.
Table 5: Status Detail Codes and Descriptions
Number
1 E_XmlValidation An XML schema validation failure occurred on the EMTS
document.
2 E_ApplicationldMapping CDX is unable to find a CDX Web user identification based on
the NAAS identification.
3 E_OtaqRegNotRegistered The CDX was unable to verify that the submitter is a registered
user in the OTAQ Registration database.
4 E_OtaqRegNotApproved The CDX was unable to verify that the submitter is approved in
the OTAQ Registration database.
37
-------
EMTS Flow Configuration Document
06/02/2010
(cont.)
Table 5: Status Detail Codes and Descriptions (cont.)
Error Status Details Value Description
Number
5 E_Distribution The CDX Node was unable to distribute the document to the
EMTS.
6 E_BackendProcessing The Document failed processing in the EMTS.
8.4 Query
The Query method is used to obtain transaction and document history.
The following URLs may be used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
Query Parameters:
securityToken: The security token issued by NAAS and returned from the Authenticate method.
dataflow: NODE
request: GetTransactionHistoryByTransactionld/GetDocuments.
parameters: See sections 5.4 and 5.5 for parameters.
Returns:
The XML representing transaction or document history. See 5.4 and 5.5 for details.
8.5 Solicit
The Solicit method is used to request EMTS documents.
The following URLs may be used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
Solicit Parameters:
securityToken: The security token issued by NAAS and returned from the Authenticate method.
dataflow: EMTS
request: GetEMTSDocument
recipient: Empty.
notificationURI: Empty.
38
-------
EMTS Flow Configuration Document
06/02/2010
Parameters:
o documentName: Mandatory parameter of string type. See Appendix F for list of document
names for the EMTS.
o organizationldentifier: Conditionally required parameter of string type. Provide an empty
string if no organization or multiple organizations are associated with the document contents.
o documentDate: Optional parameter of string type in following format: YYYY-MM-
DDThh:mm:ss±hh:mm. Leave empty for EMTS documents.
The 1.0 release of the EMTS web service documents requires the organizationldentifier be
provided.
Returns:
The CDX Node returns a StatusResponseType to the node containing the following information:
transactionld: CDX Solicit transaction ID.
status: "RECEIVED" if successful, "FAILED" if error occurs.
statusDetail: Empty.
8.6 Download
The Download method is used to download documents. The following documents can be downloaded:
Original submission
XML Validation report
QA Feedback report
Web Service documents
Archived XML files
The following URLs will be used:
Test: https://testngn.epacdxnode.net/ngn-enws20/services/NetworkNode2Service
Prod: https://cdxnodengn.epa.gov/ngn-enws20/services/NetworkNode2Service
Download Parameters:
securityToken: The security token issued by NAAS and returned from the Authenticate method.
dataflow: EMTS or EMTS-ARCHIVE
transactionld: The Transaction ID issued by the CDX Node from the Submit service.
documents: An array of NodeDocumentType. When the documents parameter is empty, the
CDX Node will return all documents associated with the Transaction ID. When a Document ID is
specified in the documents parameter, the Node will return only the associated document.
Returns:
A message contains an array of zero or more NodeDocumentType.
39
-------
EMTS Flow Configuration Document: Appendix A
06/02/2010
Node Options
There are several options for how to transport the XML file through the Exchange Network.
CDX Network Node: A server that facilitates the interface between database systems and the Exchange
Network. It is a partner's "point of presence" on the Exchange Network. Each partner has only one node.
Nodes support:
Server accessibility on the Web;
Protocols to ensure secure exchanges;
Sending and receiving standards-based messages; and
Returning requested information as XML.
Exchange Network Web Client: EPA's site for submitting environmental information via standard web
browsers. The web client supports:
Users to submit data via web-based forms and file uploads (flat file, XML file);
Users to receive submission confirmation and processing reports;
XML for the payload;
Simple Object Access Protocol (SOAP) as a wrapper for the payload;
Web Services Description Language (WSDL) for network exchange functions and services; and
Hypertext Transfer Protocol (HTTP) for secure communication via the Internet.
EMTS Web Client: The EMTS Website also supports a submittal environment which allows:
Users to submit data via file uploads (XML file only); and
XML for the payload.
A-1
-------
EMTS Flow Configuration Document: Appendix B
06/02/2010
OTAQ Registration Process
This section describes the business processes for obtaining a CDX and NAAS account and the
permissions to participate in the EMTS flow.
B-1 New User Registration
The following procedures are for all new users that are currently not registered with CDX.
All users must register for a CDX Web account by self-registering https://epacdx.net/.
Once the user has acquired a CDX Web account, they are prompted to register for one or more
programs. To register for the EMTS program, the steps are as follows:
The user registers for the EMTS flow. This will automatically register the user for the OTAQ
Registration Fuels flow.
The user enters all necessary information into OTAQ Registration Fuel application.
The user prints appropriate document prior to submitting registration information and mail the
document to the address provided on the screen.
EPA will review and approve registration materials.
User will request registration with NAAS (See section B-2).
Once approved, the user will receive permissions to submit documents to the EMTS flow and
access the EMTS website.
B-2 NAAS Node Clients User Registration
Users who plan to use Node Clients to submit files must be affiliated with the CDX Node EMTS Domain.
These users will be registered and managed by CDX Node Administrator. Setting access privileges for
the EMTS flow will be managed by the CDX Node Administrator via NAAS Web Ul. The mapping
between Web CDX user ID'S and NAAS user ID'S will be managed by the CDX Node Administrator via
NAAS application mapping Ul.
The following procedures will be required for all the users which were not registered with CDX or
registered with CDX for other than EMTS/OTAQ REG flows and does not have NAAS EMTS user ID:
1. Register with CDX Web for the EMTS flow. The user will be automatically registered for OTAQ
Registration flow.
2. Enter all necessary information into OTAQ Registration application.
3. The OTAQ Registration Approver reviews and approves the user.
4. Once approved, request the Node Administrator to register with NAAS and request privileges for
accessing EMTS flow and user mapping between the NAAS and CDX Web user ID'S. Provide
following information to the Node Administrator:
Email address to receive communications from the EMTS flow associated with the user
NAAS ID.
EMTS Web CDX user ID.
Application: EMTS
B-1
-------
EMTS Flow Configuration Document: Appendix B
06/02/2010
5. The CDX Node Administrator will register the user with NAAS. This includes:
Creating the NAAS user ID using email provided by the user.
Activating the user.
Setting privileges for the user to be able to:
o Submit EMTS data; and
o Get EMTS transaction status.
Query the CDX Node for the following:
o flow: "NODE"
o services: GetTransactionHistoryByTransactionld and GetDocuments
o parameters: Any.
o download EMTS documents.
Map the NAAS user ID to CDX Web user ID for the EMTS Application using NAAS
application mapping Ul.
Send confirmation email to the user.
For users already registered with CDX Web for the EMTS flow and who do not have NAAS user ID'S, see
sections 4.4. For all users registered with CDX Web for the EMTS flow with existing NAAS user ID'S see
sections 4.5.3 through 4.5.5.
B-3 NAAS Full Nodes User Registration
Full node users must be registered by the CDX Node Administrator. Setting access privileges for EMTS
and mapping NAAS user ID and CDX web user ID is managed by CDX Node Administrator.
The following procedures demonstrate how all users which were not registered with CDX Web or are
registered with CDX Web for flows other than EMTS/OTAQ REG and do not have NAAS EMTS user ID.
1. Register with CDX Web for the EMTS flow. The user will be automatically registered for OTAQ
Reg flow. (See Appendix B-1 for details).
2. Enter all necessary information into OTAQ Registration application.
3. The OTAQ Registration Approver reviews and approves the user.
4. Once approved, request the Node Administrator to register with NAAS and request privileges for
accessing EMTS flow and user mapping between the NAAS and CDX Web user ID'S. The user
must provide the following information to the Node Administrator:
Email address to receive communications from the EMTS flow associated with the user
NAAS ID;
- EMTS Web CDX user ID; and
Identify the application as EMTS.
5. The CDX Node Administrator will register the user with NAAS. This includes:
Creating the NAAS user ID using email provided by the user.
Activating the user.
Setting privileges for the user to be able to:
B-2
-------
EMTS Flow Configuration Document: Appendix B
06/02/2010
o Submit EMTS data.
o Get EMTS transaction status.
o Query the CDX Node for the following:
flow: NODE
services: GetTransactionHistoryByTransactionld/GetDocuments
parameters: Any
download EMTS documents.
Map the NAAS user ID to CDX Web user ID for the EMTS Application using NAAS
application mapping Ul.
Send confirmation email to the user.
For all the users already registered with CDX Web for EMTS Flow and does not have NAAS user ID'S,
see sections 4.4 through 4.8. For all the users registered with CDX Web for EMTS Flow with existing
NAAS user ID see sections 4.5.3 through 4.8.
B-4 NAAS User Revocation
If a user is deactivated from OTAQ Registration application, the user's privileges to access EMTS flow are
also revoked. The following describes the steps taken to revoke a users privileges.
1. The OTAQ Reg Approver will send CDX Node Administrator email to revoke user privileges
providing the user NAAS user ID.
2. The CDX Node Administrator will revoke the following user privileges:
Submit EMTS data.
Get EMTS transaction status.
Query THE CDX Node for the following:
o flow: NODE
o services: GetTransactionHistoryByTransactionld/GetDocuments
o parameters: Any.
Download EMTS documents.
3. The CDX Node Administrator will email OTAQ Reg Approver upon completion of revocation.
B-3
-------
EMTS Flow Configuration Document: Appendix C
06/02/2010
C
C-1 Procedure for Testing a Node in the Testing Environment
The following describe how a user should test their node in the testing environment.
1. Contact EMTS Support at support@epamts-support.com and request a pre-production NAAS Id.
You will be assigned a temporary NAAS ID in which to test the Authenticate and Submit web
services.
2. You will be provided with "test" organization and facility registered data that you will use to create
your test XML files for your file submission testing in the test environment. Do not submit any
other XML files in the test environment as the files provided have CDX web users specifically
mapped to the test NAAS ID you have been provided.
3. At a minimum, test both the Authenticate service and the Submit service. Contact EMTS support
when you have completed these two tests. EMTS Support will verify that the submitted file has
been received and processed correctly in the staging environment.
4. Upon successful completion, EMTS Support will direct you to the CDX node help desk with
instructions for setting up production NAAS credentials.
C-2 Procedure for Acquiring Credentials in the Production
Environment
In order to test node submissions in the production environment, a submitter for your organization
will need to have a valid NAAS ID and CDX web account.
1. Contact the CDX Node Help Desk (nodehelpdesk@epacdx.net') to request a production
NAAS ID, and provide the following:
The CDX Transaction ID of one Node Submission that successfully completed processing by
EMTS in the stage/test CDX environment
The preproduction NAAS ID used to conduct the test (i.e., iordanqrev@epa.gov'). CDX Node
Help Desk will verify the test submission was successful.
Request that a production NAAS ID be created and mapped to your production CDX
account. Your production CDX account must also be registered and approved in OTAQREG
prior to providing this information (see appendix B for further information on the OTAQ
Registration process). Provide your production CDX account to the CDX Node Help Desk so
that they may cross-map these two accounts.
Request that the prod NAAS ID have all access privileges (Submit, GetStatus, Solicit, Query,
and Download.)
2. The CDX Node help desk will provide the production NAAS ID to you.
C-1
-------
EMTS Flow Configuration Document: Appendix D
06/02/2010
Accessing EMTS from MyCDX Web
This section describes how users access the EMTS website from MyCDX Web. During the EMTS beta
testing period, select users will have access to the EMTS application. Users may only access the EMTS
through a registered account profile. An example of an account profile with access to the EMTS is shown
below in Figure D-1.
Figure D-1: MyCDX Account Profile
Central Data E
Welcome,
Ms. Cindy Ferreira , None
CDX Registration Status: Active
x c h a n g e - MyCDX
Last Login: December21,2009
Registered Since: November 23, 2009
Re certification Date: November 23, 2009
You have 1 new message in your Inbox
Change System Edit Personal
Password Information
Available Account Profiles;
* OTAQEMTS: OTAQ EMTS Application
OTAQREG: Fuels Proarams Reaistration
Edit Current Add New
Account Profiles Employer Profile
The EMTS URL link will be active for all registered OTAQ Fuels Registration participants. During the beta
testing period, active beta testers will have access to the EMTS by clicking on the OTAQEMTS:OTAQ
EMTS Application link. Users that are not approved beta testers will receive an error message indicating
that you are not a registered EMTS user.
After the end of the beta testing period, all users that have registered for EMTS and Fuels and have been
approved by EPA to participate in the EMTS flow can access the EMTS website through this link. Delays
may occur for new users waiting for approval of their registration materials by EPA.
D-1
-------
EMTS Flow Configuration Document: Appendix E
06/02/2010
Notifications for EMTS Flow
The following table outlines the notifications that are sent from either the CDX Node or the EMTS to the
submitter.
Table E-1: Emails Sent Regarding File Submission Status
CDX
Submitter,
CDX Help
Desk
CDX Node Failure to
Distribute Document to
EMTS.
CDX
Submitter
CDX Node XML Schema
Validation Failure on EMTS
Document.
CDX Node failed to distribute the following
document to the EMTS.
EMTS submission with transaction ID of: On:
Submission time:
Document name:
Submitter NAAS user ID:
Submitter CDX Web user ID:
Submission status: FAILED.
CDX Node may be experiencing some
technical difficulties. Please resubmit your
document at a later time. Contact the CDX
Help Desk at if the problem
persists.
Document: did not pass
XML schema validation.
EMTS submission with transaction ID of: On:
Submission time:
Submitter NAAS user ID:
Submitter CDX Web user ID:
Submission status: FAILED.
The following XML error report is available for download.
Please check the format of your XML
document against the EMTS schema to ensure
that it is valid and well-formed before re-
submitting your file.
(cont.)
Table E-1: Emails Sent Regarding File Submission Status (cont.)
E-1
-------
EMTS Flow Configuration Document: Appendix E
06/02/2010
CDX Submitter CDX is unable to Find CDX
Web User Identification
based on NAAS
Identification.
CDX Node was unable to map NAAS user ID
to CDX Web user ID.
EMTS submission with transaction ID of: On:
Submission time:
Submitter NAAS user ID:
Submitter CDX Web user ID:
Submission status: FAILED.
EMTS CDX
CDX
Submitter
EMTS has completed is
processing of the document.
CDX was unable to verify
that the submitter is a
registered user in
OTAQReg.
Please contact the CDX Node help desk at
to verify that your NAAS
account has been completed properly.
EMTS submission with transaction ID of:
Document:
Submission status: COMPLETED/FAILED.
CDX is unable to verify that you are a
registered user in the OTAQ Registration
system with permission to submit documents
to the EMTS.
EMTS submission with transaction ID of:
Document:
Submission time:
Submitter NAAS user ID:
Submitter CDX Web user ID: for
assistance.
(cont.)
E-2
-------
EMTS Flow Configuration Document: Appendix E
06/02/2010
Table E-1: Emails Sent Regarding File Submission Status (cont.)
CDX Submitter
CDX Node has completed
transaction.
EMTS submission of:
Document:
Submission time:
Submitter NAAS user ID:
Submitter CDX Web user ID: did not pass
XML schema validation.
EMTS submission with transaction ID of: On:
Submission time:
Submitter NAAS user ID:
Submitter CDX Web user ID: is available for download.
Please check the format of your XML
document against the EMTS schema to ensure
that it is valid and well-formed before
resubmitting your file.
(cont.)
E-3
-------
EMTS Flow Configuration Document: Appendix E
06/02/2010
Table E-1: Emails Sent Regarding File Submission Status (cont.)
EMTS All Submitters
with an
Organization
EMTS Processing Failure.
The following document submitted on
on behalf of could not be
processed due to one or more critical errors.
Information regarding the document is shown
below.
File name:
Submission Date/Time:
Submitter:
If you are a node user, you may retrieve the
full QA Feedback report from the CDX node
using the document download service. If you
are a web user, you may view the feedback
results online or download the document from
the EMTS website. Additional information
regarding your submission is provided below.
Submission Comment:
The following individuals were sent a copy of
this notification:
For additional support regarding the EMTS,
please contact or
.
(cont.)
E-4
-------
EMTS Flow Configuration Document: Appendix E
06/02/2010
Table E-1: Emails Sent Regarding File Submission Status (cont.)
EMTS
All Submitters EMTS Daily Processed The following documents submitted on
with an Document Summary. on behalf of have been
Organization processed.
File name:
Submission Date/Time:
Submitter:
Submission Status:
Submission Comment:
Submission Date/Time:
Submitter:
Submission Status:
Submission Comment:
{Show only if
SubmissionCommentText is not null}
If you are a node user, you may retrieve the
full QA Feedback report from the CDX node
using the document download service. If you
are a web user, you may view these results
online or download the document from the
EMTS website.
The following individuals were sent a copy of
this notification:
For additional support regarding the EMTS,
please contact or
(cont.)
E-5
-------
EMTS Flow Configuration Document: Appendix E
06/02/2010
Table E-1: Emails Sent Regarding File Submission Status (cont.)
EMTS All Submitters
within an
Organization
EMTS Notification of
Pending Trades.
Your organization, , has been
identified as a trading partner for trades. Please accept or deny
these trades no later than .
EMTS All Submitters
with an
Organization
and all
submitters with
the Trading
Partner.
EMTS Blocked Party -
Trade Not Processed
EMTS All Submitters
associated
with an
Organization
EMTS Submission Level
Failure
The following individuals were sent a copy of
this notification:
For additional support regarding the EMTS,
please contact or
.
The following trade has not been processed.
The RINs identified in this trade originated
from a source that is on the blocked list for
.
Submission Date: