Talend Cloud – Battle of the Engines

Cloud vs. Remote

To be honest, it’s not really a fight. After all they both play in the same team and can be used hand in hand. But what are the differences between both variants and when to use what engine?
To be really able to judge you need to understand what makes a Cloud Engine and what makes a Remote Engine.

Cloud Engine – Plug & Play

As a proud owner of a Talend Cloud subscription you are provided at least one Cloud Engine or more, depending on your contract. Your Cloud Engines will be completely preconfigured so you won’t have to overcome any technical obstacles. You can start right away. Talend charges you with so called tokens. Providing a Cloud Engine costs you 45000 tokens. Cloud Engines are hosted in Amazon Web Services (AWS). It is planned to have your Cloud Engines in Microsoft Azure as well by the end of this year (2019). There are no other costs for you next to the 45000 tokens. It seems obvious that due to this fact there must be some restrictions when using the cloud engines. This is simply due to the fact that Talend won’t be able to provide unlimited computing power after paying this one time fee.
As the name indicates, the cloud engine runs in the Talend Cloud. Its data is being exchanged between the cloud engine and the connected cloud SaaS services. Communication and orchestration is directly handled in the cloud between the TMC (Talend Management Console) and the cloud engine.

Cloud Engine facts

FeaturesFacts
(+) comes preconfigured Maximum memory of 525 GB
(+) AWS Cloud (Azure to be released soon) Temporary space of 200 GB
(+) for Data Integration2 VCPU’s
(+) Maximum of 3 jobs in parallelMaximum of 225 GB per
hour
(-) Talend API support not yet
implemented
Maximum of 3 jobs in
parallel
(-) Pipeline Designer support not yet
implemented
5000 tokens per engine

Remote Engine – the Allrounder

The small brother or more likely the taller one? “Small” if you take into consideration that you “only” need 9000 tokens for a remote engine. “Tall” if you think of what the remote engine is capable of. What the cloud engine is not able to cover is easily executed by the remote engine. Being sold often as a hybrid-approach in order to serve the needs of the local requirements of the customers and to simplify the transition into the cloud, it is becoming an allrounder. With being available in the AWS as well as in the Azure marketplaces, the remote engine can be used in nearly every setting, may it be on-premise or in the public cloud. The usage of a remote engine is then only limited by the available compute power. The remote engine is available for Windows as well as for Linux run systems. Other costs only occur for the provided infrastructure, whether it be the customer’s data center or – in case of a public cloud – the provided virtual machine. The remote engine can be operated from behind a firewall. Its data is only being exchanged between remote engine and its connected services. Communication and orchestration is carried out over a secured https connection to the Talend Cloud, wherewith your data is secure.

Remote Engine facts

FeaturesFacts
(-) needs to be installed and configuredOnly limited by the server’s capacity
(+) platform independent 9000 tokens per engine
(+) Unlimited amount of parallel jobs
(+) Talend API Support
(+) Pipeline Designer Support
(+) Available on AWS and Azure marketplace

Cloud, Remote or Hybrid – a choice that will help you

How does the customer benefit from having the choice and what are Talend’s hopes in offering the two engines? It sounds very much like a rivalry between these two engines. Why should I use the Cloud Engine if the remote engine can do most of it even better? Simply because: For one, Talend creates the highest level of flexibility for any use case and in any phase of a company’s transformation. For two, Talend covers a broad spectrum of customer requirements, no matter if you are already in a cloud, you are just starting or you are planning to move to the cloud at a later point in time.

Different benefits for different situations

You don’t use any cloud services by now but like to develop into a serverless direction?

With a hybrid solution you can both access your existing infrastructure via the Remote Engine as well as taking first steps towards a serverless architecture with the Cloud Engine.

You already use one of the big players in the cloud business?

With the help of a cloud engine you can start right away with implementing your requirements in the cloud. Nevertheless, a remote engine can also be a viable option due to being scalable when you need it.

You use cloud and on-premise services?

Also in that case a hybrid approach in the Talend Cloud would be the right option with offering sufficient security for your (local) data. At the same time it offers a seamless transition into a serverless architecture, as soon as older infrastructures are about to be replaced. With Cloud Engines or Remote Engines in a Public Cloud also all other imaginable scenarios are covered.

You are currently happy with your on-premise solution but might want to change later?

Here the Remote Engine can be recommended. Your services will still run locally and no customer data will be exchanged between the Talend Cloud and the Remote Engine. Only the communication and orchestration, e.g. logging or status messages, getting transfered through a secure HTTPS connection.

You move data between Cloud Services?

In this case the Cloud Engine is a simple solution. If you have to deal with more complex requirements then the Remote Engine in the Public Cloud will fullfill your needs.

You would like to combine Cloud Services with On-Premise Services?

Talend Cloud can also cover complex use cases between Remote and Cloud Engines. You might want to create procedures that automatically fetch data from a SaaS Service via the Cloud Engine. In a second step this data could be processed in a Remote Engine to, for example, combine it with Local Services, save it in a Data Warehouse and provide it again with an API.

An easy way to the future

Cloud and Remote Engine – both got its benefits and added values and they are optionally combinable. Also are they easily configured with a Talend Cloud subscription and ready to implement in nearly every infrastructure. This leads to a maximum flexibility in meeting todays and future challenges for your company.