• Accueil
• Actualités
• Logiciels
• Emplois
• Blogs
• A propos

Why use Event Sourcing?

[20/02/10]

Udi and I agree on probably 95% of what we talk about, one of the places that we have differing opinions is in the use of Event Sourcing I use the term as described previously to mean the rebuilding of objects based on events, not the definition that is currently on the bliki. To me this is an important distinction and I figured it would be worthwhile to write a post on why I feel the way I do, I explained parts of it in the previous post about CQRS and Event Sourcing but I wanted to talk not just about how the patterns are symbiotic but also some of the other reasons I use event sourcing.

 

Using a RDBMS

To start with let’s go through the alternative architecture, that is to rebuild objects from something that saves current state on the write side. I say something because that something could be many things; bigtable, mongodb, a relational database, xml files, photos of the objects that are then scanned in … It really doesn’t matter for the sake of this discussion, what matters is the storing of current state. For the sake of discussion let’s imagine that it is a relational database which Udi generally recommends here is the slide he generally uses to talk about it.

 

 

Data is stored, as normal in the relational database and the domain is instrumented to send events as well which can then be used with read model. There are some really good things about this architecture that I would like to go through before I talk about some of the issues I have run into with it in the past.

 

First of all this architecture is highly applicable to legacy systems that would like to move to a separated read model for some of their data (it is important to note that you may not move all of your data to a separated read model). it is also very familiar to development teams, operations, and management. We are well aware of how to deal with such a system, never underestimate the value of familiarity.

 

There are however some issues that exist with using something that is storing a snapshot of current state. The largest issue revolves around the fact that you have introduced two models to your data. You have an event model and a model representing current state. If you look in the above diagram you will see there are two operations, write and publish.

 

Any time that you represent things in more than one model you have to worry about keeping those two models in sync and this situation does not escape that. As an example, how do you rationalize that the data you saved to your database actually matches up with the events that you sent out? One can write unit tests to help try to keep things synchronized but they will eventually fall out of sync with each other and when they do, you have a problem.

 

Another problem with the having of two models is that it is necessarily more work. One must create the code to save the current state of the objects and one must write the code to generate and publish the events. No matter how go about doing these things it cannot possibly be easier than only publishing events, even if you had something that made storing current state completely trivial to say a document storage, there is still the effort of bringing that into the project.

 

Beyond all of that for me, the focus tends to be on the current state based model when the system is really event centric. This may sound like a nitpicky issue but I find teams doing this look at events with less importance than those who use events as storage as well as the latter only use events they are extremely event centric.

 

Using Event Sourcing

Using Event Sourcing does not change the architecture above much, the primary difference is that we have an Event Store holding the events to rebuild an object behind the domain as opposed to something storing the current state. There are however many interesting differences between the two architectures.

 

The first major difference is that it solves the problem of having two models. It removes the cost of having to deal with synchronization between the two and most importantly it removes the possibility that the two diverge. Having models diverge could be very bad as we use the event model as our integration model so others can create parallel models of our model. If we have a synchronization issue, they will have bad data while we have correct data, ouch. With Event Sourcing we only save the event (that can even be called “publishing it”)

 

There are however other benefits to using Event Sourcing as opposed to storing current state. I went through some of them briefly in another post. One of those benefits is that we can avoid having to use a 2pc transaction between the data model and the message queue (if we are using one)… the reason for this is that the event storage itself is also a queue, we could be trailing the event storage to place the items on the queue (or directly use the event storage as a queue).

 

Testing however is a big win with Event Sourcing, since all of your state changes are done through events you can simply test the events coming out the other side. This is particularly interesting when one considers that with events since you are being explicit about what is changing you are also testing what is not happening. Very few people write tests to show what doesn’t happen on behaviors (also a prime place where models can lose sync) eg: I am calling ScheduleAppointment on an object, do you check to make sure the address hasn’t changed? Since I in testing would assert that I only received an AppointmentScheduledEvent. Beyond that since we only deal with events on the other side we can view (and test) our domain as being a rather complex finite state machine which offers some very cool possibilities.

 

Of course we have left out what I think is one of the key values of Event Sourcing, we actually have the changes. Let’s say I get a concurrency violation (request is from version 5 while the current data is at version 12). With Event Sourcing I can ask for all of those events in between and see if any of those actually conflict with the command that I want to run (I will write a post on more about this works soon). In other words we provide intelligent merging…

 

You will note that I have not talked about any of the general benefits of Event Sourcing, such as the business value of the events, the value of having a log, the fact that the Event Store is additive only. I am leaving these out because many of these can exist in both systems. In the former you can save all of your events historically as well.

 

I hope this explains a bit about the differences between the models. I want to provide people as well a quick list of some of the pros and cons of each.

 

Current State Backing

Pros

Easier to retrofit on a legacy project

Well known technology/tools/skill sets

Easier sell for an organization

Cons

Dual models cost more and contain risk

Non-Event Centric

Event Storage Backing

Pros

Single Event Centric Model

Simplified/Better Testing

Conflict Management

Cons

Harder to sell to an organization

Less known tools/technologies (though you can implement the Event Store in a RDBMS which kind of mitigates this)

Very difficult to migrate a legacy app to

 

Hope this helps people.

Why use Event Sourcing?

Archives

Web Forms Routing in ASP.NET 4

[28/02/10]

At our first Sarasota Web Developer Group meeting we discussed several of the new enhancements in ASP.NET 4 Web Forms. One of my favorite enhancements is the new routing features which are very similar to the ones I have enjoyed so much in ASP.NET MVC.

Register Routes

This is old hat for those using ASP.NET MVC. Just register your routes at application startup. Rather than your endpoint being a controller, however, you associate a physical page as the handler of the request.

public class Global : System.Web.HttpApplication
{
    void Application_Start(object sender, EventArgs e)
    {
        RegisterRoutes(RouteTable.Routes);
    }

    void RegisterRoutes(RouteCollection routes)
    {
        routes.MapPageRoute(
            "Contact_Details",                  // Route Name
            "Contacts/Details/{id}",          // Url and Parameters
            "~/Contacts/Details.aspx"     // Page Handling Request
            );
    }
}

In this case we are telling the routing engine 3 things:

• Name of the Route: Contact_Details
• The Route: Contacts/Details/{id}
• The Physical Page Handling the Request: Details.aspx

Notice the id parameter ( route value ) which will be the id of the contact to display in the details page.

Expression Builders for Creating HyperLinks, etc.

With ASP.NET MVC we have strongly-typed View Helpers to help generate links. With ASP.NET 4 Web Forms you utilize Expression Builders to create links such as:

<asp:HyperLink
    NavigateUrl="<%$RouteUrl:RouteName=Contact_Details, id=1 %>"
    runat="server">John Doe
</asp:HyperLink>

<asp:HyperLink
    NavigateUrl="<%$RouteUrl:id=1%>"
    runat="server">John Doe
</asp:HyperLink>

We can explicitly specify the name of the route or let the routing engine figure out the correct route based on the parameters.

Getting RouteData from the Page

You can access the RouteData from the Page by accessing the Page.RouteData Property, which is just a convenient access point to RequestContext.RouteData. Here are a couple of ways to get the id from the route to display the proper contact given its id:

public partial class Details : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        var id = Page.RouteData.GetRequiredString("id");

        var id2 = Page.RouteData.Values["id"];
    }
}

RouteParameter for use with DataSources

If you are displaying the contact in a DetailsView, for example, you can use the new RouteParameter with your DataSource to get values from the route as such:

<asp:ObjectDataSource ID="ObjectDataSource1" runat="server" 
    SelectMethod="FindById" TypeName="Contact">
    <SelectParameters>
        <asp:RouteParameter Name="id" RouteKey="id" Type="Int32" />
    </SelectParameters>
</asp:ObjectDataSource>

Binding your DetailsView to the ObjectDataSource will now cause the contact to be displayed appropriately.

Response.RedirectToRoute and RedirectToRoutePermanent

A lot has been mentioned about using Response.RedirectPermanent for SEO, but even cooler is Response.RedirectToRoute and Response.RedirectToRoutePermanent for working with the new routing engine. Below I am specifying the route name and passing in any route values where necessary when redirecting:

Response.RedirectToRoute("Contact_Details", new { id = 1 });

Response.RedirectToRoutePermanent("Contact_Details", new { id = 1 });

Conclusion

Lots of really neat things in ASP.NET 4 and ASP.NET 4 Web Forms. I am going to continue to post a few more we discussed during the first meeting.

For those interested, the second meeting of the Sarasota Web Developer Group will discuss a number of interesting topics: Leveraging ASP.NET MVC - Web Forms - DynamicData - Castle ActiveRecord.

David Hayden

Introduction to the Reactive Extensions for JavaScript – Creating Observers

[27/02/10]

Looking back to the previous post, we covered how we create observable sequences, the producers of our data.  We have quite a number of ways of creating these outside of events which we covered earlier.  Now that we have these observable sequences, now what?  We need to address the consumer side of this producer/consumer story in the form of an observer.

Before we get started, let’s get caught up to where we are today:

• Introduction
• Creating Observables

Creating Observers

Let’s go back to the Observer pattern definition once again before we get started.  The idea here is that we have an object, called the Observable (or Subject) which keeps a list of its dependents, the observers, and notifies each of them automatically of any state changes.  In the case of the Reactive Extensions for JavaScript, we’re talking more about observable sequences.  As we discussed last time, the Observer has three parts:

• OnNext – when a new value is produced
• OnError – when an exception occurs
• OnCompleted – when the observable sequence terminates

When creating an observer, we should take all three into account and how we’re going to handle them. 

In order to attach these observers to our observable sequence, we can invoke the Subscribe method on our observable while passing in our observer.  And when we’re no longer interested in the subscription to the observable sequence, we can detach by calling Dispose on the result of the Subscribe method.

New Observer via Create

Let’s get started in creating an Observer by looking at the Observer.Create method.  This method takes in three functions, one for the OnNext, one for the OnError and finally one for the OnCompleted.  This function returns to us an Observer which we can then use for subscribing.

Rx.Observer.Create(
    function(next) { ... }, // OnNext
    function(err)  { ... }, // OnError
    function()     { ... }) // OnCompleted
);

Once we have an Observer, we can then attach to the Observable using the Subscribe method which takes our Observer.  When we call Subscribe, we get back a disposable object with a single Dispose method which allows us to detach from the Observable.

Observable {
    Subscribe : function(observer) { ... }
}

One of the best ways I find to explore a new API is to write tests to show the expected behavior.  By writing these, I have a comprehensive view of what each method does, especially if the code didn’t come with the tests already.  So, let’s create a few tests to show the behavior of creating an Observer and then subscribing to an Observable sequence.  I’ll use QUnit to write my tests, and in particular, the asynchronous test feature due because we are testing asynchronous callbacks.

The first test will be to check the OnNext function parameter on Observable.Create.  In this case, I’ll assert at the single value in my observable sequence is the value I receive when OnNext is invoked.

asyncTest("Observer should observe OnNext", function() {
    var observable = Rx.Observable.Return(0);

    var observer = Rx.Observer.Create(
        function(next) {
            equals(0, next);
            start();
        },
        function(err) { },
        function() {});

    observable.Subscribe(observer);
});

The next test, I will make an example on how my OnError function parameter will work.  In this case, I’ll have an Observer throw an exception via the Throw method and my OnError function check the message and assert that it’s the same as my error that I threw.

asyncTest("Observer should observe OnError", function() {
    var someError = "FAIL!";

    var observable = Rx.Observable.Throw(someError);

    var observer = Rx.Observer.Create(
        function(next) { },
        function(err) {
            equals(someError, err);
            start();
        },
        function() {});

    observable.Subscribe(observer);
});

Finally, in my last example, let’s create a simple test to show off the OnCompleted behavior.  In order to do so we’ll create an empty observable which should not yield any values and instead only invoke the OnCompleted.  Then we’ll create an Observer which has the test logic in the OnCompleted function parameter.

asyncTest("Observer should observe OnCompleted", function() {
    var observable = Rx.Observable.Empty();
    
    var observer = Rx.Observer.Create(
        function(next) { },
        function(err) { },
        function() {
            ok(true, "True when invoked on complete");
            start();
        });

    observable.Subscribe(observer);
});

Creating Observers this way is good for reusability, especially if you wish to attach to any number of observable sequences.  But, we’re not tied to having to create them via Create, there are other ways.

Overloading Subscribe

In addition to creating an Observer via the Create method, we also have shortcuts which allow us to create an Observer on the fly with the Subscribe method.  In addition to the Subscribe which takes an Observer, we have three other overloads which can take functions for our OnNext, OnError and OnCompleted.  The first overload takes a function for OnNext, where the next takes a function for OnNext and OnError, and finally the last overload takes functions for all three, the OnNext, OnError and OnCompleted.

Observable {
    Subscribe : function(
                    function(next) { ... })

    Subscribe : function(
                    function(next) { ... },
                    function(err)  { ... })            
                    
    Subscribe : function(
                    function(next) { ... },
                    function(err)  { ... },
                    function()     { ... })
}

This function, just as above, will create a disposable object for us which allows us to unsubscribe at any time via the Dispose method.

Unsubscribing

As I’ve stated earlier, one of the great things about the design of Rx for JavaScript is that it’s quite easy to both subscribe and unsubscribe from an observer.  The design of Rx for JavaScript follows very closely to the design of Rx for .NET including subscribing and unsubscribing.  Let’s step through an example of how we can use the Dispose method on our subscription.  In this instance, we’ll have two observers, and after the first value has been produced, we unhook the first observer and continue listening on the second.  We’ll assert that indeed the first has been unhooked while the second continues to listen.

asyncTest("Dispose should unhook observer", function() {

    var nextValue = 0;
    var observable = Rx.Observable.FromArray([1,2,3]);

    var disp1 = observable.Subscribe(
        function(next) {
            nextValue = next;
        });
    var disp2 = observable.Subscribe(
        function(next) {
            disp1.Dispose();
            equals(1, nextValue);
            start();
        });
});

Such scenarios could be quite helpful in unhooking events when others happen, such as mouse events, keyboard or even AJAX requests.  We’ll cover some of those scenarios in upcoming posts.

Conclusion

So, now we’ve covered the basics of creating Observable sequences and Observers and subscriptions.  Now that we have some of the basics what else can we do with it?  That’s where some of the LINQ combinators come in handy and we’ll pick that up next time.

This of course is only scratching the surface of what capabilities this library has and there is much more yet left to cover.  The question you’re probably asking now is where can I get it?  Well, for that you’ll have to stay tuned.  I hope to have more announcements soon about its general availability.

What can I say?  I love JavaScript and very much looking forward to the upcoming JSConf 2010 here in Washington, DC where the Reactive Extensions for JavaScript will be shown in its full glory with Jeffrey Van Gogh.  For too many times, we’ve looked for the abstractions over the natural language of the web (HTML, CSS and JavaScript) and created monstrosities instead of embracing the web for what it is.  With libraries such as jQuery and indeed the Reactive Extensions for JavaScript gives us better tools for dealing with the troubled child that is DOM manipulation and especially events.

Certified Scrum Product Owner Training class on May 24, 2010 in Boulder.

[26/02/10]
Certified Scrum Product Owner Training class on May 24, 2010 in Boulder.

[26/02/10]  - The usefulness of interaction tests, or “How to question the method”

[26/02/10]  - Castle Project code organization

[26/02/10]  - Scrum Challenge #1 OVER: Scrum is…

[25/02/10]  - A Vision for FubuMVC’s Component Model (gems, Nu, engines, slices, oh my…)

[25/02/10]  - The Exec Problem

[25/02/10]  - 3 Days To Go!

[25/02/10]  - Random Thought Scrum Challenge – #1

[24/02/10]  - DevTeach Toronto 2010 Ultimate Edition

[23/02/10]  - Introduction to the Reactive Extensions for JavaScript – Creating Observables

[23/02/10]  - Sarasota Web Developer Group - MVC and ASP.NET 4 From Scratch

[22/02/10]  - How I Stole an Office and Fixed Our Daily Scrum

[22/02/10]  - CST Application Process Improvement Community

[22/02/10]  - Scrum Alliance National, USA: Managing Director

[22/02/10]  - 10 Things I Hate About Agile Development!

[21/02/10]  - XP Day Suisse, Geneva, 29 March 2010

[21/02/10]  - In Praise of Middle Management

[20/02/10]  - Why use Event Sourcing?

• Scrum master
• Scrum tools
• Scrum sprints
• Ken Schwaber scrum
• Formation scrum

• Agile software developement scrum
• Scrum backlog
• Scrum impediment
• sprint backlog
• Product BackLog

• Backlog template
• Scrum developement
• Scrum methodologie
• Scrum certification
• Scrum meeting

• Scrum tools
• Scrum definition
• Agile
• Scrum project management
• Agile project management

• About Us
• Contact Us
• Privacy Policy
• Terms & Conditions