SimpleDB Id Column

I am having trouble with the SimpleDB DataProvider where the model has the Identity Column Name  as "LineID" in the model

when I do an insert I get

"The request must contain the parameter ItemName"

In SimpleDB, the LightSpeed Id always maps to the SimpleDB item name.  You can't remap it to a SimpleDB attribute.  So you can't use Identity Column Name with SimpleDB.  Sorry.

We should probably add a validation to detect this and produce a meaningful error -- I've logged a feature request for this.

Thats what I thought. It was thorn in my side for a 75000+ row SQL table that I was syncing to SimpleDb. It just so happens that this table had the Id column as LineId so the UnitofWork shared between MS SQL and Simpledb had that property set.

I ended up having to change the Id column on the MS SQL table. luckily it was not a live database with existing apps pointing to it. (well at least not anymore) I better work fast and get the new SimpleDB backend up before they need that database.. good thing I am the boss. I dont know why I am still typing I have work to do.

Just a quick performance test result, I usually dont care but this is huge.

Quering a Local SQL Database with Lightspeed for 77,292 entities and

inserting those 77,292 entities into SimpleDb with LightSpeed took 5+ hours

vs.

Quering Local SQL Database with Lightspeed for 77,292 entities and

converting to custom DTO and

inserting 77,292 serialized Documents(representing the entities) into MongoHQ (hosted mongodb) using NoRM took < 8 minutes

each entity had 38 properties;

Hi Tommy,

Yep - thats because currently each entity is inserted via a seperate PutAttributes call so thats the dramatic effect of latency. We are having a look at optimizing this with the BatchPutAttributes call where there is comminality on the attributes being inserted (will generally be the case!) - I will let you know once we have made some progress here :)

Jeremy

Each Entity was saved to the MongoDB one at a time Save(Enity). I checked calling Save(Enity) and instantly the entity was in the mongodb. No batch updates. Maybe the mongo driver does inserts async?

Just as an update, we have put in a new option to tonights nightly build for SimpleDB to use batching for inserts if you specify the following connection string option:

Enable Batch Insert=true

The maximum batch size supported by SimpleDB is 25, so we bundle the requests by domain name into batches of 25 if this option is on.

So this should give you a ~25x speed perf for inserts. We will have a look at applying this to the update requests in the near future as well.

Jeremy