AWS Step Functions is a fully managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Instead of writing a Decider program, you define state machines in JSON. AWS customers should consider using Step Functions for new applications. If Step Functions does not fit your needs, then you should consider Amazon Simple Workflow (SWF). Amazon SWF provides you complete control over your orchestration logic, but increases the complexity of developing applications. You may write decider programs in the programming language of your choice, or you may use the Flow framework to use programming constructs that structure asynchronous interactions for you. AWS will continue to provide the Amazon SWF service, Flow framework, and support all Amazon SWF customers. Show
Q: What are the benefits of designing my application as a coordination of tasks? How does Amazon SWF help me with this? To coordinate the application execution across workers, you write a program called the decider in your choice of programming language. The separation of processing steps and their coordination makes it possible to manage your application in a controlled manner and give you the flexibility to deploy, run, scale and update them independently. You can choose to deploy workers and deciders either in the cloud (e.g. Amazon EC2 or Lambda) or on machines behind corporate firewalls. Because of the decoupling of workers and deciders, your business logic can be dynamic and you application can be quickly updated to accommodate new requirements. For example, you can remove, skip, or retry tasks and create new application flows simply by changing the decider. By implementing workers and deciders, you focus on your differentiated application logic as it pertains to performing the actual processing steps and coordinating them. Amazon SWF handles the underlying details such as storing tasks until they can be assigned, monitoring assigned tasks, and providing consistent information on their completion. Amazon SWF also provides ongoing visibility at the level of each task through APIs and a console. Q: What can I do with Amazon SWF?
Customers have used Amazon SWF to build applications for video encoding, social commerce, infrastructure provisioning, MapReduce pipelines, business process management, and several other use cases. For more details on use cases, please see What are some use cases that can be solved with SWF?. To see how customers are using Amazon SWF today, please read our case studies. Q: What are the benefits of Amazon SWF vs. homegrown solutions and existing workflow products? Existing workflow products often force developers to learn specialized languages, host expensive databases, and give up control over task execution. The specialized languages make it difficult to express complex applications and are not flexible enough for effecting changes quickly. Amazon SWF, on the other hand, is a cloud-based service, allows common programming languages to be used, and lets developers control where tasks are processed. By adopting a loosely coupled model for distributed applications, Amazon SWF enables changes to be made in an agile manner. Q: What are workers and deciders? You can have several concurrent runs of a workflow on Amazon SWF. Each run is referred to as a workflow execution or an execution. Executions are identified with unique names. You use the Amazon SWF Management Console (or the visibility APIs) to view your executions as a whole and to drill down on a given execution to see task-level details. Q: What programming conveniences does Amazon SWF provide to write applications? Q: How is Amazon SWF different from Amazon SQS? Both Amazon SQS and Amazon SWF are services that facilitate the integration of applications or microservices:
The following are the main differences between Amazon SQS and Amazon SWF:
While you can use Amazon SQS to build basic workflows to coordinate your distributed application, you can get this facility out-of-the-box with Amazon SWF, alongside other application-level capabilities. We recommend trying both Amazon SQS and Amazon SWF to determine which solution best fits your needs. Q: What are some use cases that can be solved with Amazon SWF? Amazon SWF has been applied to use cases in media processing, business process automation, data analytics, migration to the cloud, and batch processing. Some examples are: Use case #1: Video encoding using Amazon S3 and Amazon EC2. In this use case, large videos are uploaded to Amazon S3 in chunks. The upload of chunks has to be monitored. After a chunk is uploaded, it is encoded by downloading it to an Amazon EC2 instance. The encoded chunk is stored to another Amazon S3 location. After all of the chunks have been encoded in this manner, they are combined into a complete encoded file which is stored back in its entirety to Amazon S3. Failures could occur during this process due to one or more chunks encountering encoding errors. Such failures need to be detected and handled. With Amazon SWF: The entire application is built as a workflow where each video file is handled as one workflow execution. The tasks that are processed by different workers are: upload a chunk to Amazon S3, download a chunk from Amazon S3 to an Amazon EC2 instance and encode it, store a chunk back to Amazon S3, combine multiple chunks into a single file, and upload a complete file to Amazon S3. The decider initiates concurrent tasks to exploit the parallelism in the use case. It initiates a task to encode an uploaded chunk without waiting for other chunks to be uploaded. If a task for a chunk fails, the decider re-runs it for that chunk only. The application state kept by Amazon SWF helps the decider control the workflow. For example, the decider uses it to detect when all chunks have been encoded and to extract their Amazon S3 locations so that they can be combined. The execution’s progress is continuously tracked in the Amazon SWF Management Console. If there are failures, the specific tasks that failed are identified and used to pinpoint the failed chunks. Use case #2: Processing large product catalogs using Amazon Mechanical Turk. While validating data in large catalogs, the products in the catalog are processed in batches. Different batches can be processed concurrently. For each batch, the product data is extracted from servers in the datacenter and transformed into CSV (Comma Separated Values) files required by Amazon Mechanical Turk’s Requester User Interface (RUI). The CSV is uploaded to populate and run the HITs (Human Intelligence Tasks). When HITs complete, the resulting CSV file is reverse transformed to get the data back into the original format. The results are then assessed and Amazon Mechanical Turk workers are paid for acceptable results. Failures are weeded out and reprocessed, while the acceptable HIT results are used to update the catalog. As batches are processed, the system needs to track the quality of the Amazon Mechanical Turk workers and adjust the payments accordingly. Failed HITs are re-batched and sent through the pipeline again. With Amazon SWF: The use case above is implemented as a set of workflows. A BatchProcess workflow handles the processing for a single batch. It has workers that extract the data, transform it and send it through Amazon Mechanical Turk. The BatchProcess workflow outputs the acceptable HITs and the failed ones. This is used as the input for three other workflows: MTurkManager, UpdateCatalogWorkflow, and RerunProducts. The MTurkManager workflow makes payments for acceptable HITs, responds to the human workers who produced failed HITs, and updates its own database for tracking results quality. The UpdateCatalogWorkflow updates the master catalog based on acceptable HITs. The RerunProducts workflow waits until there is a large enough batch of products with failed HITs. It then creates a batch and sends it back to the BatchProcess workflow. The entire end-to-end catalog processing is performed by a CleanupCatalog workflow that initiates child executions of the above workflows. Having a system of well-defined workflows enables this use case to be architected, audited, and run systematically for catalogs with several million products. Use case #3: Migrating components from the datacenter to the cloud. Business critical operations are hosted in a private datacenter but need to be moved entirely to the cloud without causing disruptions. With Amazon SWF: Amazon SWF-based applications can combine workers that wrap components running in the datacenter with workers that run in the cloud. To transition a datacenter worker seamlessly, new workers of the same type are first deployed in the cloud. The workers in the datacenter continue to run as usual, along with the new cloud-based workers. The cloud-based workers are tested and validated by routing a portion of the load through them. During this testing, the application is not disrupted because the workers in the datacenter continue to run. After successful testing, the workers in the datacenter are gradually stopped and those in the cloud are scaled up, so that the workers are eventually run entirely in the cloud. This process can be repeated for all other workers in the datacenter so that the application moves entirely to the cloud. If for some business reason, certain processing steps must continue to be performed in the private data center, those workers can continue to run in the private data center and still participate in the application. See our case studies for more exciting applications and systems that developers and enterprises are building with Amazon SWF. Q: Does Amazon use Amazon SWF for its own applications? Getting StartedQ: How can I get started with Amazon SWF? Q: Are there sample workflows that I can use to try out Amazon SWF? Q: What are the different ways to access SWF?
FunctionalityQ: What is registration? Q: What are domains? Q: How can I manage my application resources across different environments and groupings? Q: How does a decider coordinate a workflow in Amazon SWF? To help the decider in making decisions, SWF maintains an ongoing record on the details of all tasks in an execution. This record is called the history and is unique to each execution. A new history is initiated when an execution begins. At that time, the history contains initial information such as the execution’s input data. Later, as workers process activity tasks, Amazon SWF updates the history with their input and output data, and their latest state. When a decider gets a decision task, it can inspect the execution’s history. Amazon SWF ensures that the history accurately reflects the execution state at the time the decision task is issued. Thus, the decider can use the history to determine what has occurred in the execution and decide the appropriate next steps. Q: How do I ensure that a worker or decider only gets tasks that it understands?
Based on the above, you control which task list a task gets added into and who asks for tasks from each list. Thus, you can ensure that workers and deciders only get the tasks that they understand. Q: What is the AWS Flow Framework? How does it help me with coordinating my workflow? AWS Flow Framework makes it convenient to express both facets of coordination through familiar programming concepts. For example, initiating an activity task is as simple as making a call to a method. AWS Flow Framework automatically translates the call into a decision to initiate the activity task and lets Amazon SWF assign the task to a worker, monitor it, and report back on its completion. The framework makes the outcome of the task, including its output data, available to you in the code as the return values from the method call. To express the dependency on a task, you simply use the return values in your code, as you would for typical method calls. The framework’s runtime will automatically wait for the task to complete and continue your execution only when the results are available. Behind the scenes, the framework’s runtime receives worker and decision tasks from Amazon SWF, invokes the relevant methods in your program at the right times, and formulates decisions to send back to Amazon SWF. By offering access to Amazon SWF through an intuitive programming framework, the AWS Flow Framework makes it possible to easily incorporate asynchronous and event driven programming in the development of your applications. Q: How do workers and deciders communicate with Amazon SWF? Isn’t a poll protocol resource-intensive? To overcome the inefficiencies inherent in polling, Amazon SWF provides long-polling. Long-polling significantly reduces the number of polls that return without any tasks. When workers and deciders poll Amazon SWF for tasks, the connection is retained for a minute if no task is available. If a task does become available during that period, it is returned in response to the long-poll request. By retaining the connection for a period of time, additional polls that would also return empty during that period are avoided. With long-polling, your applications benefit with the security and flow control advantages of polling without sacrificing the latency and efficiency benefits offered by push-based web services. Q: Can I use an existing web service as a worker? Q: Does Amazon SWF restrict me to use specific programming languages? Q: I want to ensure that there is only one execution for each activation of my business process (e.g. a transaction, a submission, or an assignment). How do I accomplish this? Once the workflow execution is complete (either successfully or not), you can start another workflow execution with the same ID. This causes a new run of the workflow execution with the same execution ID but a different run ID. The run ID is generated by Amazon SWF and multiple executions that have the same workflow execution ID can be differentiated by the run ID. By allowing you to reuse workflow execution IDs in such a manner, Amazon SWF allows you to address use cases such as retries. For example, in the above user registration example, assume that the workflow execution failed when creating a database record for the user. You can start the workflow execution again with the same execution ID (user’s email address) and do not have to create a new ID for retrying the registration. Q: How does Amazon SWF help with scaling my applications? Q: I run a large number of mission critical application executions. How can I monitor and scale them? Q: I have numerous executions running at any time, but a handful of them often fail or stall. How can I detect and troubleshoot these problematic executions? To find executions that may be stalled, you can start with a time-based search to hone in on executions that are running longer than expected. Next, you can inspect them to see task level details and determine if certain tasks have been running too long or have failed, or whether the decider has simply not initiated tasks. This can help you pinpoint the problem at a task-level. Q: I have an activity type that can be used in multiple applications. Can I share it across these applications? Security and ReliabilityQ: Can I use AWS Identity and Access Management (IAM) to manage access to Amazon SWF? Q: Can I run my workers behind a firewall? Q: Isn’t it a security risk to expose my business logic as workers and deciders? Q: How does Amazon SWF help in coordinating tasks reliably in my application? LimitsQ: How many workflow types, activity types, and domains can I register with Amazon SWF? Q: Are there limits on the number of workflow executions that I can run simultaneously? Q: How long can workflow executions run? Q: What happens if my workflow execution is idle for an extended period of time? Q: How long can a worker take to process a task? Q: How long can Amazon SWF keep a task before a worker asks for it? Q: Can I schedule several activity tasks by issuing one decision? Q: How many worker tasks, signals, and markers can I have in a workflow execution and across executions? Q: How much data can I transfer within a workflow execution? Q: Does Amazon SWF retain completed executions? If so, for how long? Amazon SWF retains the history of a completed execution for any number of days that you specify, up to a maximum of 90 days (i.e. approximately 3 months). During retention, you can access the history and search for the execution programmatically or through the console. Q: When are API calls throttled? Access and AvailabilityQ: Which regions is Amazon SWF available in? Q: Is Amazon SWF available across availability zones? Q: What are the Amazon SWF service access points? BillingQ: Do your prices include taxes? Except as otherwise noted, our prices are exclusive of applicable taxes and duties, including VAT and applicable sales tax. For customers with a Japanese billing address, use of AWS services is subject to Japanese Consumption Tax. Learn more. What term is used to describe a repeated sequence of production activities?Mass production is also referred to as flow production, repetitive flow production, series production, or serial production. In mass production, mechanization is used to achieve high volume, detailed organization of material flow, careful control of quality standards, and division of labor.
Which of the following is likely to increase the utilization of a resource with a setup time?Which of the following is likely to increase the utilization of a resource with a setup time? Increasing the batch size on a resource with setups always increases the capacity of the process.
What shape is used to represent resources in a process flow diagram?The ____ time of a resource is how long that particular resource takes to complete one flow unit. What shape is used to represent resources in a process flow diagram? -Triangled.
|