Package org.apache.beam.sdk.io.gcp.pubsub

Class PubsubTestClient

java.lang.Object

org.apache.beam.sdk.io.gcp.pubsub.PubsubClient

org.apache.beam.sdk.io.gcp.pubsub.PubsubTestClient

All Implemented Interfaces:

java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable

public class PubsubTestClient
extends PubsubClient
implements java.io.Serializable

A (partial) implementation of PubsubClient for use by unit tests. Only suitable for testing publish(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath, java.util.List<org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.OutgoingMessage>), pull(long, org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, int, boolean), acknowledge(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, java.util.List<java.lang.String>) and modifyAckDeadline(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, java.util.List<java.lang.String>, int) methods. Relies on statics to mimic the Pubsub service, though we try to hide that.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	PubsubTestClient.PubsubTestClientFactory	Closing the factory will validate all expected messages were processed.

Nested classes/interfaces inherited from class org.apache.beam.sdk.io.gcp.pubsub.PubsubClient

PubsubClient.IncomingMessage, PubsubClient.OutgoingMessage, PubsubClient.ProjectPath, PubsubClient.PubsubClientFactory, PubsubClient.SchemaPath, PubsubClient.SubscriptionPath, PubsubClient.TopicPath

Constructor Summary

Constructors

Constructor	Description
PubsubTestClient()

Method Summary

Modifier and Type	Method	Description
int	ackDeadlineSeconds(PubsubClient.SubscriptionPath subscription)	Return the ack deadline, in seconds, for subscription.
void	acknowledge(PubsubClient.SubscriptionPath subscription, java.util.List<java.lang.String> ackIds)	Acknowldege messages from subscription with ackIds.
void	advance()	For subscription mode only: Track progression of time according to the Clock passed .
void	close()
static PubsubTestClient.PubsubTestClientFactory	createFactoryForCreateSubscription()
static PubsubTestClient.PubsubTestClientFactory	createFactoryForGetSchema(PubsubClient.TopicPath expectedTopic, @Nullable PubsubClient.SchemaPath expectedSchemaPath, @Nullable org.apache.beam.sdk.schemas.Schema expectedSchema)
static PubsubTestClient.PubsubTestClientFactory	createFactoryForPublish(@Nullable PubsubClient.TopicPath expectedTopic, java.lang.Iterable<PubsubClient.OutgoingMessage> expectedOutgoingMessages, java.lang.Iterable<PubsubClient.OutgoingMessage> failingOutgoingMessages)	Return a factory for testing publishers.
static PubsubTestClient.PubsubTestClientFactory	createFactoryForPull(com.google.api.client.util.Clock clock, PubsubClient.SubscriptionPath expectedSubscription, int ackTimeoutSec, java.lang.Iterable<PubsubClient.IncomingMessage> expectedIncomingMessages)	Return a factory for testing subscribers.
static PubsubTestClient.PubsubTestClientFactory	createFactoryForPullAndPublish(PubsubClient.SubscriptionPath pullSubscription, PubsubClient.TopicPath publishTopicPath, com.google.api.client.util.Clock pullClock, int pullAckTimeoutSec, java.lang.Iterable<PubsubClient.IncomingMessage> expectedIncomingMessages, java.lang.Iterable<PubsubClient.OutgoingMessage> expectedOutgoingMessages, java.lang.Iterable<PubsubClient.OutgoingMessage> failingOutgoingMessages)	Returns a factory for a test that is expected to both publish and pull messages over the course of the test.
void	createSchema(PubsubClient.SchemaPath schemaPath, java.lang.String schemaContent, com.google.pubsub.v1.Schema.Type type)	Create Schema from Schema definition content.
void	createSubscription(PubsubClient.TopicPath topic, PubsubClient.SubscriptionPath subscription, int ackDeadlineSeconds)	Create subscription to topic.
void	createTopic(PubsubClient.TopicPath topic)	Create topic.
void	createTopic(PubsubClient.TopicPath topic, PubsubClient.SchemaPath schema)	Create {link TopicPath} with PubsubClient.SchemaPath.
void	deleteSchema(PubsubClient.SchemaPath schemaPath)	Delete PubsubClient.SchemaPath.
void	deleteSubscription(PubsubClient.SubscriptionPath subscription)	Delete subscription.
void	deleteTopic(PubsubClient.TopicPath topic)
org.apache.beam.sdk.schemas.Schema	getSchema(PubsubClient.SchemaPath schemaPath)	Return a Beam Schema from the Pub/Sub schema resource, if exists.
PubsubClient.SchemaPath	getSchemaPath(PubsubClient.TopicPath topicPath)	Return PubsubClient.SchemaPath from PubsubClient.TopicPath if exists.
boolean	isEOF()	Return true if PubsubClient.pull(long, org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, int, boolean) will always return empty list.
boolean	isTopicExists(PubsubClient.TopicPath topic)	Return true if topic exists.
java.util.List<PubsubClient.SubscriptionPath>	listSubscriptions(PubsubClient.ProjectPath project, PubsubClient.TopicPath topic)	Return a list of subscriptions for topic in project.
java.util.List<PubsubClient.TopicPath>	listTopics(PubsubClient.ProjectPath project)	Return a list of topics for project.
void	modifyAckDeadline(PubsubClient.SubscriptionPath subscription, java.util.List<java.lang.String> ackIds, int deadlineSeconds)	Modify the ack deadline for messages from subscription with ackIds to be deadlineSeconds from now.
int	publish(PubsubClient.TopicPath topic, java.util.List<PubsubClient.OutgoingMessage> outgoingMessages)	Publish outgoingMessages to Pubsub topic.
java.util.List<PubsubClient.IncomingMessage>	pull(long requestTimeMsSinceEpoch, PubsubClient.SubscriptionPath subscription, int batchSize, boolean returnImmediately)	Request the next batch of up to batchSize messages from subscription.

Methods inherited from class org.apache.beam.sdk.io.gcp.pubsub.PubsubClient

createRandomSubscription, extractTimestampAttribute, parseTimestampAsMsSinceEpoch, projectPathFromId, projectPathFromPath, schemaPathFromId, schemaPathFromPath, subscriptionPathFromName, subscriptionPathFromPath, topicPathFromName, topicPathFromPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PubsubTestClient

public PubsubTestClient()

Method Detail

createFactoryForPublish

public static PubsubTestClient.PubsubTestClientFactory createFactoryForPublish(@Nullable PubsubClient.TopicPath expectedTopic,
                                                                               java.lang.Iterable<PubsubClient.OutgoingMessage> expectedOutgoingMessages,
                                                                               java.lang.Iterable<PubsubClient.OutgoingMessage> failingOutgoingMessages)

Return a factory for testing publishers. Only one factory may be in-flight at a time. The factory must be closed when the test is complete, at which point final validation will occur.

createFactoryForPull

public static PubsubTestClient.PubsubTestClientFactory createFactoryForPull(com.google.api.client.util.Clock clock,
                                                                            PubsubClient.SubscriptionPath expectedSubscription,
                                                                            int ackTimeoutSec,
                                                                            java.lang.Iterable<PubsubClient.IncomingMessage> expectedIncomingMessages)

Return a factory for testing subscribers. Only one factory may be in-flight at a time. The factory must be closed when the test in complete

createFactoryForPullAndPublish

public static PubsubTestClient.PubsubTestClientFactory createFactoryForPullAndPublish(PubsubClient.SubscriptionPath pullSubscription,
                                                                                      PubsubClient.TopicPath publishTopicPath,
                                                                                      com.google.api.client.util.Clock pullClock,
                                                                                      int pullAckTimeoutSec,
                                                                                      java.lang.Iterable<PubsubClient.IncomingMessage> expectedIncomingMessages,
                                                                                      java.lang.Iterable<PubsubClient.OutgoingMessage> expectedOutgoingMessages,
                                                                                      java.lang.Iterable<PubsubClient.OutgoingMessage> failingOutgoingMessages)

Returns a factory for a test that is expected to both publish and pull messages over the course of the test.

createFactoryForGetSchema

public static PubsubTestClient.PubsubTestClientFactory createFactoryForGetSchema(PubsubClient.TopicPath expectedTopic,
                                                                                 @Nullable PubsubClient.SchemaPath expectedSchemaPath,
                                                                                 @Nullable org.apache.beam.sdk.schemas.Schema expectedSchema)

createFactoryForCreateSubscription

public static PubsubTestClient.PubsubTestClientFactory createFactoryForCreateSubscription()

advance

public void advance()

For subscription mode only: Track progression of time according to the Clock passed . This will simulate Pubsub expiring outstanding ACKs.

close

public void close()

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

publish

public int publish(PubsubClient.TopicPath topic,
                   java.util.List<PubsubClient.OutgoingMessage> outgoingMessages)
            throws java.io.IOException

Description copied from class: PubsubClient

Publish outgoingMessages to Pubsub topic. Return number of messages published.

Specified by:

publish in class PubsubClient

Throws:

java.io.IOException

pull

public java.util.List<PubsubClient.IncomingMessage> pull(long requestTimeMsSinceEpoch,
                                                         PubsubClient.SubscriptionPath subscription,
                                                         int batchSize,
                                                         boolean returnImmediately)
                                                  throws java.io.IOException

Description copied from class: PubsubClient

Request the next batch of up to batchSize messages from subscription. Return the received messages, or empty collection if none were available. Does not wait for messages to arrive if returnImmediately is true. Returned messages will record their request time as requestTimeMsSinceEpoch.

Specified by:

pull in class PubsubClient

Throws:

java.io.IOException

acknowledge

public void acknowledge(PubsubClient.SubscriptionPath subscription,
                        java.util.List<java.lang.String> ackIds)
                 throws java.io.IOException

Description copied from class: PubsubClient

Acknowldege messages from subscription with ackIds.

Specified by:

acknowledge in class PubsubClient

Throws:

java.io.IOException

modifyAckDeadline

public void modifyAckDeadline(PubsubClient.SubscriptionPath subscription,
                              java.util.List<java.lang.String> ackIds,
                              int deadlineSeconds)
                       throws java.io.IOException

Description copied from class: PubsubClient

Modify the ack deadline for messages from subscription with ackIds to be deadlineSeconds from now.

Specified by:

modifyAckDeadline in class PubsubClient

Throws:

java.io.IOException

createTopic

public void createTopic(PubsubClient.TopicPath topic)
                 throws java.io.IOException

Description copied from class: PubsubClient

Create topic.

Specified by:

createTopic in class PubsubClient

Throws:

java.io.IOException

createTopic

public void createTopic(PubsubClient.TopicPath topic,
                        PubsubClient.SchemaPath schema)
                 throws java.io.IOException

Description copied from class: PubsubClient

Create {link TopicPath} with PubsubClient.SchemaPath.

Specified by:

createTopic in class PubsubClient

Throws:

java.io.IOException

deleteTopic

public void deleteTopic(PubsubClient.TopicPath topic)
                 throws java.io.IOException

Specified by:

deleteTopic in class PubsubClient

Throws:

java.io.IOException

listTopics

public java.util.List<PubsubClient.TopicPath> listTopics(PubsubClient.ProjectPath project)
                                                  throws java.io.IOException

Description copied from class: PubsubClient

Return a list of topics for project.

Specified by:

listTopics in class PubsubClient

Throws:

java.io.IOException

isTopicExists

public boolean isTopicExists(PubsubClient.TopicPath topic)
                      throws java.io.IOException

Description copied from class: PubsubClient

Return true if topic exists.

Specified by:

isTopicExists in class PubsubClient

Throws:

java.io.IOException

createSubscription

public void createSubscription(PubsubClient.TopicPath topic,
                               PubsubClient.SubscriptionPath subscription,
                               int ackDeadlineSeconds)
                        throws java.io.IOException

Description copied from class: PubsubClient

Create subscription to topic.

Specified by:

createSubscription in class PubsubClient

Throws:

java.io.IOException

deleteSubscription

public void deleteSubscription(PubsubClient.SubscriptionPath subscription)
                        throws java.io.IOException

Description copied from class: PubsubClient

Delete subscription.

Specified by:

deleteSubscription in class PubsubClient

Throws:

java.io.IOException

listSubscriptions

public java.util.List<PubsubClient.SubscriptionPath> listSubscriptions(PubsubClient.ProjectPath project,
                                                                       PubsubClient.TopicPath topic)
                                                                throws java.io.IOException

Description copied from class: PubsubClient

Return a list of subscriptions for topic in project.

Specified by:

listSubscriptions in class PubsubClient

Throws:

java.io.IOException

ackDeadlineSeconds

public int ackDeadlineSeconds(PubsubClient.SubscriptionPath subscription)
                       throws java.io.IOException

Description copied from class: PubsubClient

Return the ack deadline, in seconds, for subscription.

Specified by:

ackDeadlineSeconds in class PubsubClient

Throws:

java.io.IOException

isEOF

public boolean isEOF()

Description copied from class: PubsubClient

Return true if PubsubClient.pull(long, org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath, int, boolean) will always return empty list. Actual clients will return false. Test clients may return true to signal that all expected messages have been pulled and the test may complete.

Specified by:

isEOF in class PubsubClient

createSchema

public void createSchema(PubsubClient.SchemaPath schemaPath,
                         java.lang.String schemaContent,
                         com.google.pubsub.v1.Schema.Type type)
                  throws java.io.IOException

Description copied from class: PubsubClient

Create Schema from Schema definition content.

Specified by:

createSchema in class PubsubClient

Throws:

java.io.IOException

deleteSchema

public void deleteSchema(PubsubClient.SchemaPath schemaPath)
                  throws java.io.IOException

Delete PubsubClient.SchemaPath.

Specified by:

deleteSchema in class PubsubClient

Throws:

java.io.IOException

getSchemaPath

public PubsubClient.SchemaPath getSchemaPath(PubsubClient.TopicPath topicPath)
                                      throws java.io.IOException

Description copied from class: PubsubClient

Return PubsubClient.SchemaPath from PubsubClient.TopicPath if exists.

Specified by:

getSchemaPath in class PubsubClient

Throws:

java.io.IOException

getSchema

public org.apache.beam.sdk.schemas.Schema getSchema(PubsubClient.SchemaPath schemaPath)
                                             throws java.io.IOException

Description copied from class: PubsubClient

Return a Beam Schema from the Pub/Sub schema resource, if exists.

Specified by:

getSchema in class PubsubClient

Throws:

java.io.IOException