Monday, December 14, 2009

Eclipselink: table sequence generator and concurrency - Part Two, Custom Sequence generator

In my previous post I explained how EclipseLink TableGenerated primary keys have a problem with concurency. I'm not sure why and whether other people had the same problem, but what can you do? Well, we had to do something.

First, I cosidered possibility of different configuring of primary key generation straegy for different platforms, using master entity superclass and confguring with XML descriptors (actually combining XML descriptors and annotations).

The, I tried to find some solution that enables us to somehow make EclipseLink use some alternative method of primary key generation. Fortunately, it did not take long.

On EclipseLink wiki there is an example of custom sequencing using UUID. Now, UUID generation is something we came across while using Hibernate, who has some issues using HashMaps and identifying objects, so generating UUID used in isEqual methods for objects is very recommender. Using UUID as primary key sounded interesting, but, frankly, I did not feel like modifying database schema...

So I wondered, why not making my own, concurrency safe version of table sequencing? After all, it is very nice solution, takes my mind of thinking whether I use MS, Oracle or whatever DB and I dont bother worrying over whether identity columns will do just as good job as Oracle sequences...

Let's look into customizing sequencing.

You can extend Eclipselink classes Sequence and StandardSequence. You also have to implenent SessionCustomizer interface.

public class UUIDSequence extends Sequence implements SessionCustomizer{

Now, look into the class. For us important methods are

public Object getGeneratedValue(Accessor accessor,
AbstractSession writeSession, String seqName) {
return UUID.randomUUID().toString().toUpperCase();
}
and

public void customize(Session session) throws Exception {
UUIDSequence sequence = new UUIDSequence("system-uuid");

session.getLogin().addSequence(sequence);
}

getGeneratedValue method, well, returns the generated sequence value in whatever way we want. In the example UUID is used, so simply radnom generated uuid value is returned from the method.

But, how do we make Eclipselink use our sequence generator? customize method is used for this. As you can see, a new instance of our sequence class is created with our specific name given to constructor, and our sequence is added to session. To tell eclipselink that we are using our custom generator we add a property in the persistence unit

 name="eclipselink.session.customizer" 
value
="eclipselink.example.UUIDSequence"/>
Where value of the property is full qualified class name of our Sequencing class.

And that is it. I'm not really sure how this works, but persistence unit sees that it has customizable class, method customize is called and our sequence is set. When you want to use sequence in your JPA entities, you siply specify name you passed to constructor

@Id
@GeneratedValue(generator="system-uuid"/>
@Column(name="PROJ_ID")
private int id;

In the next post we will make similar class which will use our own table generated sequence. And yes, it is quite simple :)

Wow, not bad post after 10 days of flu...

Wednesday, November 25, 2009

Eclipselink: table sequence generator and concurrency - Part One

Why do we like OR mapping? OK, you got tons of advertising written about that, so take your pick.
For your boss most likely most interesting thing would be universal database connectivity. You have customers with various databases, so it is much easier to sell your software. You dont have to buy a licence for a new database, relax, we got them all covered... Actually, that was our case.

OK, that is cool. We had our little piece of software up and running on Eclipselink (I had quite an adventure on migrating from Hibernate, but that is another story). Since we wanted to be database vendor independent, we chose table generated ID's for our primary keys.

Little intro, just to put some light on it, but I'm sure you guru's out there don't need it ;) Here is a good place to start. There you can find details on ID generation strategies.

When you specify an entity in JPA, you have to set one of the class fields as an ID, using annotation @Id.

So the next step is to see how to assign those ID's. You could set values yourself before persisting an object, but that is not very useful in practice. There are also options for using identity or sequencing, but this is not applicable on all databases. For instance, identity columns are available in SQL Server, but Oracle uses sequences. If you want universal solution, portable on all databases, you need table generated primary keys. It is similar to Oracle sequences, only the values are stored in a user table Eclipselink is accessing. If you use schema generation, Eclipselink will, according to your annotations, generate that table. Usually table has columns for sequence name and value, allocation size (default 50). If you look into the documentation on @TableGenerator annotation, you will see that you can configure name for this table, its columns, and also values for allocation size etc.

So it all sounded nice. And on Hibernate it worked nice. But life just isn't that simple. Life is fun.

While doing some stress testing on some application modules I became aware of an unpleasant fact that unusual number of unique constraints is occuring.

What I did was that I deployed application module and set up some 5 clients on others computers to target it with a large number of subsequent requests which typically did some inserting and updating in the database. Then it turned out that I get a relatively large number of unique constraint errors - on primary keys!

So, 5 clients target EJB modules, services in modules access database using Eclipselink. New objects are generated and mapped into tables. When persisting objects, Eclipselink takes tabel generated value and assigns it as objects ID. INsert into... And same thing happens initiated from several clients. One should expect that Eclipselink gives different values for primary keys each time new object is persisted. It shoul handle concurrency. But it looks like it does not.

Of course, we have checked our test cases and they were OK. It works just fine in a single client situation. When we addes another client, only 4-5 unique constraints occured on some 10000 inserts. Adding new clients increased exoenentially number of exceptions.

I have written few words to Oracle, on official forum as well on Metalink (now Oracle Support). Well, guys told me that it it shouldn't happen. But it does. It is not anything exotic - stateless session beans, jpa persistence using eclipselink provider on Weblogic server. You look up bean, method creates and persists object. It is not clustered environment.

I would really like to know if anyone else had similar problem, so if you hear anything... :)

Then search for alternative methods for generating ID's started. After a while, it turned out that Eclipselink is flexible enough to give us opportunity to use our own ID generation mechanism.

Next post will be really interesting one, when I present our cool solution :D

Eclipselink: Null primary key encountered in unit of work clone

Remember that XKCD strip about the most boring guy in the world and his blog
where he only apologizes for not writing more? :D Here it is ;)

Well, it's been a while. To be honest, civilian military service is not actually best environment for us developers, but what can you do, from June next year, I'm back on track.

Meanwhile, a blast from the past. I received a notification about a bug on Eclipselink you can find here. Of course, I encountered it last summer. Suddenly a weird error Null primary key encountered in unit of work clone, as the title of this post (I hope it helps with google).

So, this is one of those things that just frustrates you. You develop application, write queries by the book, OK, push the limits probably on few fronts, but it's all according to specification, should work, wright? Yeah, should. But bug like this just kills you. It occurs seemingly randomly, fires up nad causes some null pointer exceptions in upper layers of your application, so you dig in and find thing like this. I can tell you, I've been lucky here. In most cases I just run (actually Google does) into two or three guys who had the same problem (usually our Corean brotheren, hats off ! ), and no solution...

So, it had something to do with assigning sequence numbers. I went so far to dig into the source code of Eclipselink, but honestly, I couldn't get my way around it, even debugging (I'm not proud) didn't help. Guys who discussed the problem on the thread about the bug suggested that the problems are negative ID's which could not occur in our application. At the end, we've given up. And now the blessed news:

Chris Delahunt 2009-11-12 16:13:34 EST
Peter checked in a fix 02 November 2009

I'll check it with the guys at work to see if they still have the same problem
as before and if new patch will help.

This reminds me of a very interesting problem tha led to very
interesting solution,regarding sequencing, concurency and
multiplatform implementation of JPA.
Interested? Well, OK, you will have it in the next blog post which,
I hope will not be seven months from now . :)



Saturday, February 14, 2009

How we handled ADF Security

Well. we're handling it, alright.

One of the things you must take into account is that developing in JDeveloper with integrated WLS has advantages, but it can be living hell when you try to move application to standalone server. I guess development environment gives you some advantages by setting some things where they should be so you can focus on programming itself. And that is cool. What is not cool is that some of the details remain hidden from you, so you can easily overlook them.

As for ADF Security and moving it to other environments, I found this very helpfull. Andrejus gives you nice step-by-step introduction, with pointing to relevant articles and tutorials from Oracle. But...

What wasn't said, or maybe I have missed it, was that you have to put adf-config.xml file on the classpath. If you download Andrejus' example, when you make a deloyment EAR you will see that there is a directory adf with META-INF directory that has connections.xml and, most important, adf-config.xml

adf-config.xml has few things you need for using ADF Security.


Of course, you can find enough documentation on this.

And this is put to our classpath by using adf-loc.jar that is in the lib directory of EAR. This jar contains only META-INF with manifest file containing this

Manifest-Version: 1.0
Class-Path: ../adf


So that does the trick.

But, now what?

First, I don't like this restarting server part. It means that changing policies during production requires migration process and restarting server, which I think no one likes. I guess that integrating previous Oracle technologies with WLS has just begun, this has to be corrected ASAP.

Second, I guess there aren't many people whith knowledge from both worlds (Oracle and BEA), which will probably change in time. Until then, we have to handle things like this.

Anyway, I would like to thank Andrejus for his article and his lovely example that showed me the light.

Cheers!

Thursday, February 12, 2009

Some trouble with AWT libraries on Linux

Our application is fine, thanks for asking. Our painfull security problem is gone, which will probably be the topic of next posto. But here is the fresh problem and a fresh solution.


Our app is using some fancy graph solutions for displaying some statistical data form the application. We are using some ADF stuff, there are some cute controls to help you with that. We are developing on Windows, so the test servers were there. So, today is the big day when I am setting up some VMs for live testing and we are using Linux on them. WLS is installed, domain configured (actually, I just copied domain folder from Windows to Linux and changed paths etc. also an interesting topic) . Server started, I go to a login page, log in, I get main window whith menus and wait for my graphs to be rendered... And the, exception printed on the console, my session breaks and I am returned to the login page!

Here are the interesting parts from stacktrace:

SEVERE: Server Exception during PPR, #1
javax.servlet.ServletException: java.lang.InternalError: Can't connect to X11 window server using 'centosbi.antegra.com:99' as the value of the DISPLAY variable

java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment

X11? WTF?


Google is my best friend, so I came up with some pointers quite fast:

link1

So, it turns out that if I use something from AWT, I need X environment. No way! Well, the guys suggested xvfb solution, which didnt make me very happy... But some guy suggested to set java parameter

-Djava.awt.headless=true

So I tried that. I set it into setDomainEnv.sh file like java property in line JAVA_OPTIONS=...


And it worked.


So, if you use Oracle graphical components for rendering bars, pie charts etc. on Linux, you better set this parameter.

Cheers!

Sunday, February 1, 2009

Basic EclipseLink persistence.xml configuration

Right,let's start with the basics.

Weblogic has inherited from it's previous versions O/R mapping called KODO. I don't have much to say about it, but it is default ORM for Weblogic, just as Toplink was for OC4J or HIbernate is for JBoss. Of course, JPA is implemented on all of these platforms, and the implementation is done in these persistence mappers. You should try to use JPA as much as possible, and use HIbernate/EclipseLink specifics only where you don't have any other choice. Simple rule, but as always, the problem is judging when to do that. We will say more about it in some other post.

Let's get back to persistence.xml. Since default ORM for Weblogic is KODO, if you try to use persistence.xml you used on OC4J,

Here we used addidtional properties for generating tables based on our JPA entities. It can be useful in development. Of course, not in production, you don't want to drop all tables when you deploy application. Note here that on Toplink we had to excplicitly say type of databse so this would work. We also specified logging level for toplink.

Weblogic will assume that it should use default ORM, KODO. Then you will see some KODO specific messages, and things will work... To a certain point.

As it turned out, regardless of JPA standard, implementations are not perfect. The last thing you need is changing ORM in the middle of the project (believe me, Hibernate, Toplink... I know!). If you developed with Toplink so far, you should stick to what you know. So we want to stick to Toplink which meanwhile became EclipseLink. OK, you get it with new Weblogic. Now we just have to put couple more things in persistence.xml






Note couple of things. First, we had to tell Weblogic what persistence provider we want to use:

org.eclipse.persistence.jpa.PersistenceProvider

Make sure you put it right after persistence-unit tag.

Second, name of eclipselink properties are equivalent to toplink properties. Where once was toplink, now is eclipselink. Same thing goes for most of the classes, package naming etc. So all of this makes transition from Toplink to Eclipselink on Weblogic quite simple. Until we get to classloading and structuring our EAR for new platform.

That is another story.

Hope this helps someone!

Saturday, January 31, 2009

Did you get used to WebLogic?

Last autumn it was quite a surprise for us when it turned out that after number of technical previews (four, actually) Oracle is giving up on OC4J and bringing BEA into its warm nest. Yeah, perhaps we should have listened better, or is it just me expecting that after TP1, TP2, TP3 and TP4 we get final release? Well, what's there is there. We had application in development which at some point took a turn from Jboss/Hibernate platform (it was quite a road, I tell you) and went Oracle style. In next series of posts we will describe in great detail what changing from on application server/persistence layer is like, and during last year we had two of them. Just when I was finishing the work on OC4J, big news of Weblogic coming to Oracle!

I remember that last summer there was a poll on Oracle forum. Question was if we started developing products with TP4. At the time about 54% of people said yes, and twenty something said that they are going to. It is good thing I didn't start writing this blog back then :D After some time, I can say that we're cool with our new app. server. It is not like we had any choice, but Weblogic is quite good.

Last few years I have been J2EE developer and I worked on Jboss and OC4J 10. So few words about them.

Jboss is cool. I did not use it in production, just in development, but it was fast enough, it is cheapest you can get, nice community, configuring is XML based, but no restarts needed.

OC4J 10. Well, at least it's got nce console... Not too nice, that default OC4J ADF look makes a man sick. Deployment was not flexible enough, speed, naah, and you needed to restart it quite more than you would like.

OK, then the previews came. 1,2,3,4... We had no graphic console, but I can't say that making command scripts really hurts. If you want point-click, you are in wrong business. But bugs... Jesus, Joe and Marry! Weird errors, null pointer exceptions when you have unique constraints, totally useless error logging... Try developing on a thing like that. You hope for a finale release, remember the old days when everything was cool. OK, not everything was cool, but I least knew what was going on.

So, what has Weblogic brought to us?

First, my favourite topic, persistence layer. We got EclipseLink now! And I am really happy! All those cool stuff we had in TopLink is now here, it's in the community, we can see source (not that I need it, but it is good feeling ;) ) , guys on the news group respond to all our stupid and not so stupid questions in reasonable time. Using it is almost out of the box, since we need to put couple more lines in our persistence.xml to override using KODO (more about this in next post), we ran into couple of bugs (easy, easy...), but I am satisfied, mostly because you can get a lot uf useful info.

I recomend this little site EclipseLink - Eclipsepedia and a news group.


Then Weblogic itself. Nice console, a lot of options for deployment, strong base of users... No wonder Oracle bought it. Not so nice thing is that old BEA forum was not avalilable when I was looking for some answers, although it was shown in Google search results, so I had to use Google cache... JMS services are more complex than we got used to. All server is, actually, around clustering and guys that were presenting us Weblogic couple of months ago were proud of that.

There are issues about compiling code (if you have generics), and structuring deployment archive. Adapting code that worked fine on Jboss and OC4J was not very pleasant thing, and the deployment archive we finally made work had quite unexpected structure. Now issues of transfering Oracle ADF security to a standalone production are main problem. We hope we will make it work... Just like everything else. I can only say for now that following instructions from Oracle site, was not enough.

Actually, it was never enough. We had to experiment a lot, got couple of times in dead end, and sometimes I had exceptions that only occured two times on Google... And no one knew answer.

I hope you had better luck. But we will be happy to share everythinig we found out so far. Maybe it will help someone (solidarity, brothers and sisters!) and perhaps some of you will have few pointers. On Monday I will start with basic persistence layer configuration and maybe few notes what doesn't work.

Well, earn your daily bread with Weblogic! It is a good life.