This week I took the time to upgrade my new project from Rails 2 to Rails 3. Thank your respective deity that I did so. There are a few complaints. I miss having to manually hack in bundler's gem management as well as the long complicated routes file. Where, I ask you, will I spend those hours debugging that used to boil down to a bad route configuration? Where? I shutter at the thought of being able to read my routes file and not have to fight with gem versions.
One thing that truly bothered me, a minor annoyance but one none the less, was that script_finder no longer works. I think I will have to fork the project and provide a Rails 3 version, that is of course, unless someone can provide me with a link to the Rails 3 friendly version.
Showing posts with label Migrations. Show all posts
Showing posts with label Migrations. Show all posts
Tuesday, December 21, 2010
Tuesday, November 9, 2010
Has and belongs to many through
While working on a legacy data conversion project I decided to create a Rails project in order to cheat so I didn't have to write a ton of SQL statements to dump things to flat files in order to operate on them. However, while doing this, I realized that the table names, and the entire model structure of the legacy application was jacked up hard.
This caused me to use has_and_belongs_to_many with the :through argument along with association_foreign_key and foreign_key. Let me tell you how stupid I felt when I realized I was looking at the wrong version docs.
Brain needs to be on before turning to Google for the answer. The code that does what I needed is below:
has_and_belongs_to_many :users, :join_table => "ib_users_accounts_link",
:foreign_key => "accounts_id",
:association_foreign_key => "user_id"
This caused me to use has_and_belongs_to_many with the :through argument along with association_foreign_key and foreign_key. Let me tell you how stupid I felt when I realized I was looking at the wrong version docs.
Brain needs to be on before turning to Google for the answer. The code that does what I needed is below:
has_and_belongs_to_many :users, :join_table => "ib_users_accounts_link",
:foreign_key => "accounts_id",
:association_foreign_key => "user_id"
Friday, October 1, 2010
Rails Join Tables
Wow,
Bad morning. Took me 30 min to find out that the reason I was throwing an error on a controler was not due to bad code, but due to a bad migration that I wrote.
When writing the join table migration I neglected to specify that there was no primary key. Oracle HATES this.
def self.up
drop_table :email_blasts_users
create_table :email_blasts_users, :id => false do |j|
j.references :email_blast
j.references :user
end
end
Make sure to have that :id => false if you don't want your database to yell at you.
Bad morning. Took me 30 min to find out that the reason I was throwing an error on a controler was not due to bad code, but due to a bad migration that I wrote.
When writing the join table migration I neglected to specify that there was no primary key. Oracle HATES this.
def self.up
drop_table :email_blasts_users
create_table :email_blasts_users, :id => false do |j|
j.references :email_blast
j.references :user
end
end
Make sure to have that :id => false if you don't want your database to yell at you.
Wednesday, September 15, 2010
VPD and Oracle Scheduled Jobs
A few months ago we had a minor problem, the companies' development Oracle server died. It did something fun and dropped a partition or two and generally went belly up. All things considered, not a major issue. The death of Oracle caused us some lost time mostly due to the fact that most of us were not running Oracle Express which would have allowed us to keep on developing even with a down Oracle cluster. The quick lesson here, run Oracle Express if your box will handle the load.
One fallout of this problem, besides the lost time was that we also lost our primary "Gold Schema." This was a big problem.... The birth of our system was a bastard child conceived by one of our Senior Developers as part of a bet. The conversation went something like this...
As any of you that have had this kind of conversation know, this was a bad idea on both sides. It lead to the creation of the new system, which was supposed to be a proof of concept, in under a week. Given it was a 60-80 hour week but all the same, code that is rushed like that takes on a ton of technical debt and tends to inherit the legacy system's debt as well. The debt I am currently talking about is that the Rails application's database was a copy of a MySQL database ported over into Oracle on short notice. Using this tactic, we do not have migrations from blank to current state. Not usually an issue with Oracle considering our in-house systems allow us to request a clone of a current schema, but in this case, with the loss of the Oracle schema, we lost our baseline.
Recovering the baseline from one of the developers that was using Oracle Express was straightforward, but we forgot one thing when we made that developers' schema the master. We forgot about VPD. For those of you that don't know what VPD is, be thankful. It is Oracle's home grown security system affectionately called Virtual Private Database. VPD has good uses, and ones that can be transparent to the developer. Things like table level security and filtering as well as constructing database sessions with additional audit information. A few examples can be found here. One it is not good at is row level security, which is what we were using it for.
Jobs are another part of the Oracle schema that we failed to remember at first. Our Database Developer informed me that there are at least three ways you can schedule jobs. We managed to not clear out all of these when we removed VPD from our security model. One of the jobs that was scheduled every fifteen minutes constructed a materialized view of what user was allowed to see what document based on a series of permission levels and tags. Generally you can imagine that this job would not cause any issues, it was just reading from around four tables and constructing a new view. All good. But in this case, things were not as nice as they should have been, you see, there was a bug.
The bug caused this job to lock all of the rows that it was using to construct the view in an attempt to verify that the view it would construct was accurate. Oracle locking somewhere in the range of half a million rows across four tables causes things like deadlocks. At least the deadlock provided a trace showing what had locks on the rows. As soon as we found this, we went through the new "gold" schema and blew away all remaining vestigial VPD operations.
Lessons to be learned:
--Just because you have a hammer does not make it the right tool for the job
One fallout of this problem, besides the lost time was that we also lost our primary "Gold Schema." This was a big problem.... The birth of our system was a bastard child conceived by one of our Senior Developers as part of a bet. The conversation went something like this...
S: I really think the attempt to upgrade the PHP application to a newer version is a waste of time.
A: I think it is better then the alternative of starting over on a new platform.
S: I bet I can get a working version of the system on Rails in a week!
A: So do it.
As any of you that have had this kind of conversation know, this was a bad idea on both sides. It lead to the creation of the new system, which was supposed to be a proof of concept, in under a week. Given it was a 60-80 hour week but all the same, code that is rushed like that takes on a ton of technical debt and tends to inherit the legacy system's debt as well. The debt I am currently talking about is that the Rails application's database was a copy of a MySQL database ported over into Oracle on short notice. Using this tactic, we do not have migrations from blank to current state. Not usually an issue with Oracle considering our in-house systems allow us to request a clone of a current schema, but in this case, with the loss of the Oracle schema, we lost our baseline.
Recovering the baseline from one of the developers that was using Oracle Express was straightforward, but we forgot one thing when we made that developers' schema the master. We forgot about VPD. For those of you that don't know what VPD is, be thankful. It is Oracle's home grown security system affectionately called Virtual Private Database. VPD has good uses, and ones that can be transparent to the developer. Things like table level security and filtering as well as constructing database sessions with additional audit information. A few examples can be found here. One it is not good at is row level security, which is what we were using it for.
Jobs are another part of the Oracle schema that we failed to remember at first. Our Database Developer informed me that there are at least three ways you can schedule jobs. We managed to not clear out all of these when we removed VPD from our security model. One of the jobs that was scheduled every fifteen minutes constructed a materialized view of what user was allowed to see what document based on a series of permission levels and tags. Generally you can imagine that this job would not cause any issues, it was just reading from around four tables and constructing a new view. All good. But in this case, things were not as nice as they should have been, you see, there was a bug.
The bug caused this job to lock all of the rows that it was using to construct the view in an attempt to verify that the view it would construct was accurate. Oracle locking somewhere in the range of half a million rows across four tables causes things like deadlocks. At least the deadlock provided a trace showing what had locks on the rows. As soon as we found this, we went through the new "gold" schema and blew away all remaining vestigial VPD operations.
Lessons to be learned:
- Make sure your migrations allow you to build a new database from scratch
- Verify that migrations remove tasks / jobs / views that are no longer needed and could impact the performance of your system
- Do not use a bet as a good reason to create a new production system
- Attempt to learn from the last generation of software's sins
- Be friends with your DBA and DBD, they can save your ass.
--Just because you have a hammer does not make it the right tool for the job
Subscribe to:
Posts (Atom)