Newscast Explorer
Technologies used
Introduction
In this project are going to explore a known EIP known as Saga.
A saga is in its essence, a way to describe a sequence of processes that should occur sequentially in a transactional way.
There are two major Saga types: A choreography and an orchestration Saga types.
A choreography Saga is based on event sourcing. We have multiple decoupled processes that wait for certain events in order to be triggered.
An orchestration Saga is also based on event sourcing, but instead of having the events scattered, it provides a series of rules on how processes should react to certain sub-process results. An orchestration Saga has provisions for rollback processes for example. As soon as an error occurred, all provisioned methods to perform a rollback will be triggered from the point of failure. This follows the direct opposite direction of the successful route. In this saga, we lay out a chain of processes and for each process we can optionally define a rollback process or any other process we want to trigger should a fail occur.
Sagas work in a different way than chained processes in the sense that they are thought out to be design to predict failures and act upon them. Whether we choose Choreography or Orchestration, we always get an out-of-the-box plan to go forward and go back.
In this project we have two steps:
Fetching data
We run a process to fetch data from a mock feeder. This is not directly related to Sagas but it is part of the example of the use case. The data is sent in a message node containing an author node. The fetch process will run by default configured take maximum 30 seconds per run and a maximum of 100 messages under that run.
Sending comments
We will send comments via POST requests. If the id's of the page, author and message match, then the sagas will run normally. If the any of the id's fail, the process will perform a reactive chain of processes which will mark these comments as not available in the database. It will perform this in this order in reverse. So this means that if the id of a page does not match, there will be a comment of that page in the database as a dangling reference. It will be marked as not available. The comment for the author and the message will be ignored because the sagas will not allow the chain to continue. If the author id does not match, both page and author comments will be marked as not available. There will be no record of message.
This repo is the official support article to my article on medium:
News Cast — Using Sagas in Choreography and Orchestration Patterns
Project Layout
- news-cast-explorer-common - Java libraries to support Page, Authors and Messages
- news-cast-explorer-saga-common - Kotlin library to support the comments for the previously mentioned entitites
- news-cast-mock - News feeder simulation. This is where the data starts - Spring - Kotlin module - Port 8081
- news-cast-explorer-fetcher - Makes different runs, scheduled or non schedule. Each run can fetch up to 100 messages in a maximum period of 30 seconds by default - Spring Java 17 module - Port 8080
- news-cast-explorer-cdc - A CDC mock service to support message exchange in the Kafka streams - Port 8085
- news-cast-explorer-saga-choreography - A choreography implementation of the Saga Architecture - Spring - Kotlin module - Port 8083
Example request:
curl -X POST http://localhost:8083/api/saga/choreography -H 'Content-Type: application/json' --data '{ "idPage": 1, "pageComment": "I love this", "idAuthor": 2, "authorComment": "This is my favourite author", "idMessage": 3, "messageComment": "I agree", "authorRequestId":123,"pageRequestId":456,"messageRequestId":789 }'
- news-cast-explorer-saga-orchestration - An orchestration implementation of the Saga Architecture - Spring - Kotlin module - Port 8082
Example request:
curl -X POST http://localhost:8082/api/saga/orchestration -H 'Content-Type: application/json' --data '{ "idPage": 1, "pageComment": "I love this", "idAuthor": 2, "authorComment": "This is my favourite author", "idMessage": 3, "messageComment": "I agree", "authorRequestId":123,"pageRequestId":456,"messageRequestId":789 }'
GUI
- news-cast-explorer-fe - A front end tool providing and interface to visualize the results of the news feed fetcher
Swagger UI
- Local run
- news-cast-explorer-fetcher
- news-cast-explorer-saga-choreography
- news-cast-explorer-saga-orchestration
- news-cast-mock
- Local Docker run
- news-cast-explorer-fetcher
- news-cast-explorer-saga-choreography
- news-cast-explorer-saga-orchestration
- news-cast-mock
How to run
make dcup-full-action
Run Cypress
Against Nginx
make cypress-open-docker
Directly against the service ports
make cypress-open
Installation Notes
Java version
sdk install java 17-open
sdk use java 17-open
Local hosts configuration
Running the containers may require you to add this line to your /etc/hosts file:
127.0.0.1 news_cast_kafka
This is mostly because of the way kafka operates in giving responses back
Roadmap
For roadmap information and current developments please check document Roadmap.md
References
- Eventuate.IO
- Saga: How to implement complex business transactions without two phase commit
- Managing data consistency in a microservice architecture using Sagas - Implementing an orchestration-based saga
- Managing data consistency in a microservice architecture using Sagas - Implementing a choreography-based saga
- Choreography pattern with Springboot
- Spock Framework Reference Documentation
- Interaction Based Testing with Spock
- Bash tips: Colors and formatting (ANSI/VT100 Control sequences)
- JUnit 5 Parameter Resolution Example
About me 👨🏽💻
🚀
🏳️🌈
50 Dec 30, 2022
7 Feb 21, 2022
3 Mar 23, 2022
546 Jan 2, 2023
58 Dec 3, 2022
2 Oct 1, 2022
0 Jan 17, 2022
3 Jul 2, 2021
4 Sep 27, 2022
288 Dec 25, 2022
188 Dec 30, 2022
7 Dec 16, 2022
2 Mar 18, 2022
2 Oct 17, 2022
4 Dec 23, 2022
5 Jul 20, 2022
2.3k Dec 30, 2022
69 Dec 21, 2022