CPu/IRP integration V1.1
Document date | Document version | Scope definition |
---|---|---|
08/08/2024 | 1.0 | CPu version 4.1.0: CPu/IRP integration |
19/09/2024 | 1.1 | Adapted roll-out plan in section 4.2. |
1. Context
1.1. Why… Approximate matching
Current implementation of CPu (ref. 4.0 Release and previous releases of the CPu platform) is facing challenges regarding matching of messages where different reference keys are used in messages by different stakeholders. This can be understood in context of go live of CPu where the barrier was lowered for stakeholders to start sending messages (such as a Terminal Discharge) to CPu with basic identifiers for a container. Since different stakeholders (e.g. Ship Agent vs. Terminal Operator vs. Customs Authority) are using different reference keys, CPu is compensating this with matching logic using a hiërarchy depending on the message and identifiers provided. This results in a system prone to errors.
One of the goals in context of the integration of CPu with IRP is eliminating this approximate matching logic and replacing this with a consistent and exact matching logic. This could be realised when an agreement is made to evolve towards a consistent and unique key to identify a container, shared between the different stakeholders using CPu. The unique reference key for a container stay which had been agreed upon when interacting with CPu is as follows:
Equipment Number, Bill Of Lading(s), Ship Agent EORI, Terminal NxtEntityId (and Stay Number)
The use of Stay Number is optional but recommended.
In section 3 the impact on message level is addressed. Above reference key which is required to be unique for an active container stay enables 100% matching level accuracy of the different messages.
1.2. Why… Waiting room matching
Another challenge regarding the matching of messages from different stakeholders is the timeframe and chronological order wherein the messages are received by CPu, which is unpredictable. The start object of the container stay in CPu is currently the Commercial Release from the Ship Agent. For business and other reasons, the Commercial Release is often granted to the First Release Party after Terminal Discharge and Customs Status Release. An example of such scenario is visualised by below diagram.
In order to mitigate this potential of receiving messages before the commercial release, a waiting room is put in place that enables out of order processing (which is currently set at 30 days). Such mechanism of compensating logic causes higher complexity of the system and makes it sensitive to errors. On top of that, the messages received are not visible to a container’s stakeholders and no notifications are sent.
2. CPu/IRP integration logic
2.1. Overview
Below diagram visualises the main interactions of a Ship Agent and Terminal Operator with NxtPort in context of the CPu/IRP integration. This diagram is not exhaustive and kept rather simple so that focus is on the impact for the main and most important interactions. Please note that other message flows can evolve in future releases.
2.2. Definitions
In context of the CPu/IRP integration, it’s relevant to define following concepts:
“Not-released” Commercial Release: This is a commercial release in CPu that is created resulting from a Terminal Discharge message. It is considered not released since the Ship Agent didn’t release it to the First Release Party. It’s not accessible via UI nor API. Instead, it’s used as a reference object for a container stay that allows to match messages with this object.
Customs light CPu: The definiton of the CSR greenlight in IRP and the Customs greenlight in CPu will be identical from the 4.1 release onwards. The definition of a GREEN Customs light in CPu changes to: “Container can leave the Terminal land- and seaside”.
Interpretation of concepts on the overview diagram in section 2.1 (Commercial Release, Terminal Discharge, Terminal Release, Validate Pick-up and Gate-out) remain identical as in release 4.0.
2.3. Matching messages: reference key
As mentioned in section 1.1, the unique reference key to identify a container stay, which must be used in all messages to CPu and IRP is:
Equipment Number and Bill Of Lading(s) and Ship Agent EORI and Terminal NxtEntityId (and Stay Number)
The use of Stay Number is optional but recommended;
If stay number is not known for a commercial release, it can’t be used as additional identifier to match messages.
If stay number is known but not provided by the terminal, it can’t be used either to match the terminal messages.
If stay number is known and provided by the terminal, it will be used to match the message. When incorrect, a validation exception is returned.
For messages sent by the Terminal Operator, Terminal NxtEntityId can be derived from identifying the entity sending the message to CPu. For messages sent by the Ship Agent, Terminal NxtEntityId can be derived from the Terminal Code provided.
2.4. Functional integration logic
The essence of the functional integration logic from CPu and IRP is that messages are matched 1 to 1 (ref. section 2.3. and 3.) and that either the Commercial Release or Terminal Discharge is the starting entry of a container stay in CPu (which allows to eliminate the waiting room).
In case the Ship Agent sends a Commercial Release for a unique reference key and there’s no Terminal Discharge yet for the corresponding key, the Commercial Release is the starting entry of the container stay in CPu. Following messages such as Terminal Discharge, Terminal Release, CSR,… for the same reference key will be linked to the existing Commercial Release and corresponding greenlights will change.
In case the Terminal Operator sends a Terminal Discharge for a unique reference key and there’s no Commercial Release yet for the corresponding key, the Terminal Discharge is the starting entry of the container stay in CPu. As a result of this Terminal Discharge message;
a “not-released” commercial release object is created in CPu,
the Terminal Discharge Greenlight will be set to “DISCHARGED” / “GREEN” and a notification for this event is sent to the Ship Agent and Terminal Operator (ref. https://documentation.nxtport.com/certified-pickup/cpu-notifications-overview#CPunotificationsoverview-“type”:“ReleaseLight”).
This “not-released” commercial release object allows that other messages (such as Terminal Release, CSR and Commercial Release) with identical reference key can be linked and eliminates the waiting room. Below diagram visualises this approach:
IRP is sharing Terminal Discharge messages with CPu for “container” goods flows within “BEANR”. All discharges from sea going vessels (including those for Transshipment containers) are shared.
A “not-released” commercial release object and all linked messages (Terminal Discharge, CSR,…) is archived when there’s no commercial release by Ship Agent within 60 days from Terminal Discharge.
The “not-released” commercial release object and its greenlights is not accessbile via UI nor via API. However, notifications for Greenlight changes (ref. https://documentation.nxtport.com/certified-pickup/cpu-notifications-overview#CPunotificationsoverview-“type”:“ReleaseLight”) are sent event based. This implies that for example a Terminal Release or Customs greenlight changed notification is sent by CPu at the moment the original messages are processed, which can be before the actual Commercial Release is “released” by the Ship Agent.
2.5. Important scenario’s
2.5.1. Container changes from “import” to “transshipment”
As a result of the definition change of Customs light in CPu (which is now aligned with the CSR from IRP for the container stay), it’s a responsibility of the Ship Agent to delete or revoke a Commercial Release in CPu when the container will leave the terminal seaside, in order to avoid unlawful withdrawal.
Example scenario to clarify:
Commercial Release given by Ship Agent
Discharge and Terminal Release by Terminal Operator
REN added to TSD by Ship Agent
CSR by Customs Authority (which will set the Customs light in CPu to “TRANSHIPMENT”)
→ Ship Agent must delete or revoke the commercial release so that the container can’t be pickup anymore land side via CPu
2.5.2. Terminal Update
Since a Terminal Discharge can create a “not-released” Commercial Release in CPu, there might be scenario’s where there’s a Commercial Release with a wrong Terminal identifier in CPu in parallel with a “not-released” Commercial Release (resulting from the Terminal Discharge). The Terminal NxtEntityId will be the sole differentiator in the reference key between both objects. This can happen for example in case the Terminal for a Commercial Release is not updated before Terminal Discharge.
In such cases, it’s important for the Ship Agent to delete the Commercial Release with wrong Terminal identifier and create a new Commercial Release with the correct Terminal identifier. The new Commercial Release with correct Terminal identifier will match with the existing “not-released” Commercial Release and the latter becomes “released”.
Please note that a Terminal update for a Commercial Release will not be possible after Terminal Discharge from version 4.1, when Terminal Operators start sending Terminal Discharges via IRP, since the reference key for a Commercial Release in CPu (“released” and “not-released”) must be unique. This resolves the issue with loss of messages (Terminal Discharge, Terminal Release, CCRM,…) that are received before the Terminal is updated for a Commercial Release (Terminal identifiers are used in matching messages ref. section 2.3 and 3).
2.5.3. Delete Commercial Release
A request to delete a commmercial release will archive the release right and its greenlights. When a new commercial release is created for the same reference key, received greenlights will remain RED with their default value “UNKNOWN” for Terminal Discharge light and “NOTRELEASED” for Customs light. This principle applies for messages received from IRP (Terminal Discharge and CSR). CCRM and NGPS are still using the out-of-order processing mechanism.
Please note that abuse of the delete feature results in data loss, examples of such scenario’s are:
Use of delete commercial release feature to update the expiry date
Instead, following feature is available: https://documentation.nxtport.com/certified-pickup/shipping-agent-integration-guidelines#ShippingAgentIntegrationSpecifications-Updatecommercialrelease
Use of delete commercial release feature to revoke a release right
Instead, following feature is available: https://documentation.nxtport.com/certified-pickup/shipping-agent-integration-guidelines#ShippingAgentIntegrationSpecifications-Revokecommercialrelease
3. Impact per functional message
3.1. TSD
Ref. IRPMIG section 15.1.1 (https://www.inboundrelease.com/nl/download-de-irpmig-en-de-irpbig)
3.2. Commercial Release
3.2.1. Message structure
There are no changes to the message structure of a commercial release. Relevant identifiers are present (ref. section 2.3).
API documentation: https://documentation.nxtport.com/certified-pickup/shipping-agent-integration-guidelines#ShippingAgentIntegrationSpecifications-Submitcommercialrelease
3.2.2. Business logic
From this moment, the commercial release is visible in CPu UI / API and the Commercial Release Greenlight is set to “RELEASED” with color “GREEN”.
3.3. Terminal Discharge
3.3.1. Message structure
Ref. IRPMIG section 15.1.4 (https://www.inboundrelease.com/nl/download-de-irpmig-en-de-irpbig)
3.3.2. Business logic
IRP is sharing Terminal Discharge messages with CPu for “container” goods flows within “BEANR”. All discharges from sea going vessels (including those for Transshipment containers) are shared. Terminal Discharges are forwarded from IRP to CPu immediately after they are received.
When there’s a Commercial Release for the corresponding reference key, the Terminal Discharge Greenlight is set to “DISCHARGED” with color “GREEN”.
When there’s no Commercial Release for the corresponding reference key, a “not-released” Commercial Release object is created (ref. section 2.4) and the Terminal Discharge Greenlight is set to “DISCHARGED” with color “GREEN”.
3.4. Terminal Release (and Block)
3.4.1. Message structure
In order to have a 1 to 1 match between a Terminal Release message and a Commercial Release in CPu (ref. section 2.3); billOfLadingNumbers
and eoriShipAgent
have been added as mandatory variables in the request for V2. Please find API documentation for V2:
Terminal Release: https://documentation.nxtport.com/certified-pickup/terminal-operator-integration-guidelines#TerminalOperatorIntegrationSpecifications-SubmitTerminalRelease
3.4.2. Business logic
The Terminal Release Greenlight is set to “RELEASED” with color “GREEN” (or “BLOCKED” with color “RED”). Please note that since the waiting room has been eliminated, a Commercial Release or Terminal Discharge for the corresponding key must be registered in CPu when a Terminal Release (or Block) is sent.
3.5. Validate Pick-up Truck
3.5.1. Message structure
In order to ensure the correct commercial release is selected when requesting Pick-up validation for Truck (ref. section 2.3);
billOfLadingNumbers
andeoriShipAgent
have been added as mandatory variables in the request for V2stayNumber
has been added as an optional and recommended identifier
Please note that in V2, pickup validation can solely be requested for a single container.
Please find API documentation for V2: https://documentation.nxtport.com/certified-pickup/terminal-operator-integration-guidelines#TerminalOperatorIntegrationSpecifications-SynchronousPick-upvalidationtruck
3.5.2. Business logic
There are no business logic changes.
3.6. Validate Pick-up Barge/Rail
3.6.1. Message structure
In order to ensure the correct commercial release is selected when requesting Pick-up validation for Barge/Rail (ref. section 2.3);
billOfLadingNumbers
andeoriShipAgent
have been added as mandatory variables in the request for V3stayNumber
has been added as an optional and recommended identifier
Please find API documentation for V3: https://documentation.nxtport.com/certified-pickup/terminal-operator-integration-guidelines#TerminalOperatorIntegrationSpecifications-SynchronousPick-upvalidationbarge&rail
3.6.2. Business logic
There are no business logic changes.
3.7. Gate Out
3.7.1. Message structure
In order to have a 1 on 1 match between a GateOut message and a Commercial Release in CPu (ref. section 2.3);
billOfLadingNumbers
andeoriShipAgent
become mandatory variables in the request for V2fullEmptyIndicator
is removedmodeOfTransport
has been added (optional)
Please find API documentation for V2: https://documentation.nxtport.com/certified-pickup/terminal-operator-integration-guidelines#TerminalOperatorIntegrationSpecifications-SubmitTerminalGateOut
3.7.2. Business logic
There are no business logic changes regarding the impact of a Gate Out message on a Commercial Release, which will be archived after gate-out.
Additional attribute modeOfTransport
has been added so that this can be used to anticipate a Next Mode of Transport project.
fullEmptyIndicator
is removed since the logic to set the Customs light to “NOTAPPLICABLE” will dissapear in next 4.2 release. For an empty container Customs Authority is currently sending CCRM messages with status “Release”, which sets the Customs light to “RELEASED” with color GREEN.
4. Implementation strategy and roll out plan
4.1. Hybrid version
Release 4.1 is intended to be a hybrid version that supports the old and new message processing flows
An example of an old processing flow;
Discharge (sent to CPu) and Terminal Release by Terminal Operator
Commercial Release given by Ship Agent
CCRM from Customs Authority
Example of a hybrid processing flow;
Discharge (sent to IRP) and Terminal Release by Terminal Operator
Commercial Release given by Ship Agent
CCRM from Customs Authority
Example of fully new processing flow;
Discharge (sent to IRP) and Terminal Release by Terminal Operator
Commercial Release given by Ship Agent
CSR from Customs Authority (sent from IRP)
The waiting room and corresponding out-of-order processing mechanism is still enabled therefore in version 4.1.0. Please note that this applies solely to messages received via the old message processing flow (e.g. CCRM, Discharge sent to CPu,…)
Messages received by CPu from IRP don’t use the out-of-order processing mechanism. As soon as all messages received by CPu are switched to the new processing flow (this mainly depends currently on development at Customs Authority), the waiting room and its out-of-order processing mechanism will be removed (release version to be determined).
4.2. Roll out plan
Release to PRD of CPu version 4.1.1 is planned on 29th of September 2024.
The dependencies on PN/TS and IRP, described in section 4.2. of “CPu/IRP integration V1.0”, can be neglected since an enable/disable feature toggle can be used for the sharing of messages from IRP to CPu. Sharing of Terminal Discharge and CSR messages from IRP to CPu will therefore be enabled only as soon as E2E testing is finished by relevant stakeholders.
With regards to deprecating V1 endpoints (and V2 for Pickup Validation Barge/Rail) ref. section 3 and according API documentation, it has to be determined with the CPu user community when this will happen.
V2 (except for Pickup Validation Barge/Rail) and V3 endpoints are available in version 4.1.1 but recommended not to be used until EORI numbers are provided by all Ship Agents ref. https://documentation.nxtport.com/certified-pickup/terminal-operator-integration-guidelines.