Examples
Several examples of Flink jobs that use different aspects of the AMPS Flink connector are located in the repository for the connector:
https://github.com/60East/amps-integration-apache-flink
tip
flink-connector-amps-examples/src/main/com/crankuptheamps/flink/example/helper/Constants.java may need to have the URI constants updated with the IP address or DNS name of the host running AMPS unless you are running both the Flink cluster and the AMPS server on the same system.
The repository includes jobs such as:
| Example Name | Demonstrates |
|---|---|
AMPSSinkExample.java | The AMPS Sink publishing messages to AMPS. |
AMPSSourceExample.java | The AMPS Source receiving messages from AMPS. |
AggregationExample.java | A job that uses Flink's aggregation and watermarks to aggregate messages from AMPS. |
BatchExample.java | A job that uses topN to demonstrate bounded/batch usage of the AMPS Source. |
BlogExample.java | Jobs that use Flink's windowing to monitor a data stream from AMPS. This example is further explored in a blog. |
CheckpointExample.java | A job that intentionally throws exceptions to demonstrate the AMPS Source and AMPS Sink delivery guarantees as well as their fault tolerance. |
CustomDeserializerExample.java | A job that uses a custom deserialization schema when deserializing messages from AMPS. |
CustomSerializerExample.java | A job that uses a custom serialization schema when serializing messages being published to AMPS. |
MessageQueueExample.java | Jobs that use message queues with the connectors. Also includes an example that demonstrates fault tolerance when using the connectors with a message queue. |
ParallelSourceExample.java | A job that highlights how messages are not lost when using parallelism with the AMPS Sources. |
PublishStoreExample.java | A job that shows how publish stores can be used by the AMPS Sink to ensure all messages are delivered. |
ReplicationExample.java | Jobs that revolve around replication with the connectors. |
SOWExample.java | A job that focuses on SOW queries. |
SSLExample.java | A job that is a modification of SimpleExample.java that involves setting up an SSLContext for the AMPS clients used by the connectors. |
SimpleExample.java | A job that uses the connectors to publish and receive ad hoc messages. |
SoakTest.java | A long-running job that uses ad hoc messages to ensure the connectors do not encounter a failure when running for long periods of time. |
TableExample.java | Jobs that involve using the Flink table API along with the connectors. |
VolumeExample.java | A job that receives a large volume of messages from an AMPS Source and publishes those messages back to AMPS with an AMPS Sink. |