Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node: latitude/longitude storage #4292

Merged
merged 3 commits into from
Dec 23, 2018
Merged

Node: latitude/longitude storage #4292

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fbd2293
populating new fields
milaaraujo Dec 18, 2018
d54e973
adding precision/scale
milaaraujo Dec 19, 2018
9869cda
updating example schema
milaaraujo Dec 19, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions db/migrate/20181219043340_add_fields_to_node.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
class AddFieldsToNode < ActiveRecord::Migration[5.2]
def up
add_column :node, :latitude, :decimal, :precision => 20, :scale => 17
add_column :node, :longitude, :decimal, :precision => 20, :scale => 17
add_column :node, :precision, :integer

sql = "SELECT `node`.`nid`,`term_data`.`name` FROM `node` LEFT OUTER JOIN `community_tags` ON `community_tags`.`nid` = `node`.`nid` LEFT OUTER JOIN `term_data` ON `term_data`.`tid` = `community_tags`.`tid` WHERE (term_data.name LIKE 'lat:%' or term_data.name LIKE 'lon:%')"
tags_array = ActiveRecord::Base.connection.exec_query(sql)
tags_array.each do |tag|
node_id = tag["nid"]
if tag["name"].include? 'lon:'
long = tag["name"].gsub("lon:","").to_f
update "UPDATE `node` SET `precision` = '" + decimals(long).to_s + "' WHERE nid = '" + node_id.to_s + "'"
update "UPDATE `node` SET `longitude` = '" + long.to_s + "' WHERE nid = '" + node_id.to_s + "'"
else
lati = tag["name"].gsub("lat:","")
update "UPDATE `node` SET `latitude` = '" + lati + "' WHERE nid = '" + node_id.to_s + "'"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @milaaraujo it looks like this didn't run correctly on production for some reason -- odd that it didn't fail on stable, but i'm looking into it now. I think in general we may want to stick to non-raw SQL in the future if possible as there may be more safeguards... but i don't really know that this is the issue right now:

Mysql2::Error: Incorrect decimal value: 'false' for column 'latitude' at row 1
ActiveRecord::StatementInvalid: Mysql2::Error: Incorrect decimal value: 'false' for column 'latitude' at row 1: UPDATE `node` SET `latitude` = 'false' WHERE nid = '2120'

I wonder if some of the lat/lon tags are not supplying numbers!!??

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha! I think this tag exists, though not sure where: https://publiclab.org/tag/lat:false

OK, i'll try to manually remove it and re-run the migration. If that doesn't work, we may want to amend the migration to only run if the latitude/longitude values are actually numbers...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, deleted lat:false as a tag. I tried scanning on the console for any other non-numerical ones and wasn't able to find any. Hopefully it'll run properly this time, restarting! Sorry for the scare! This is publishing to production now, exciting!!!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did it work?

end
end
end

def down
remove_column :node, :latitude
remove_column :node, :longitude
remove_column :node, :precision
end

def decimals(n)
if not n.to_s.include? '.'
return 0
else
return n.to_s.split('.').last.size
end
end
end
5 changes: 4 additions & 1 deletion db/schema.rb.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
#
# It's strongly recommended that you check this file into your version control system.

ActiveRecord::Schema.define(version: 2018_11_29_100000) do
ActiveRecord::Schema.define(version: 2018_12_19_043340) do

create_table "answer_selections", force: true do |t|
t.integer "user_id"
Expand Down Expand Up @@ -234,6 +234,9 @@ ActiveRecord::Schema.define(version: 2018_11_29_100000) do
t.string "slug"
t.integer "legacy_views", default: 0
t.integer "views", default: 0
t.decimal "latitude", precision: 20, scale: 17
t.decimal "longitude", precision: 20, scale: 17
t.integer "precision"
end

add_index "node", ["changed"], name: "node_changed", using: :btree
Expand Down