Skip to main content

Ruby : Code Snippets - How to Downloading 600+ files under https links

require 'nokogiri' #install gem nokigiri
require 'open-uri'
require 'openssl'

def get_page(url)
return Nokogiri::HTML(open(url, {ssl_verify_mode: OpenSSL::SSL::VERIFY_NONE}))
end

def read_file(the_html_file)
return Nokogiri::HTML(open(the_html_file))  
end

html_file = 'C:\Users\OSMCAN\Desktop\baliwtm.html'

page = read_file(html_file)   
#puts page

sources = page.css('source').map { |source| source['src'] }
#puts sources


sources.each do |url|
if url.include? "?"
url = url.split('?').first
end
puts url

the_filename = File.basename(url)
puts "the_filename: #{the_filename}"
the_url = "#{File.dirname(url)}/#{the_filename}" 
puts "the_url: #{the_url}"
open(the_filename, 'wb') do |file|
  file << open(the_url, {ssl_verify_mode: 0}).read #require 'openssl'
end
#exit
end

Comments

Popular posts from this blog

Ubuntu 16.04 LT - Installing Chromium browser

> Type these commands about this PPA : $ sudo add-apt-repository ppa:canonical-chromium-builds/stage $ sudo apt-get update $ sudo apt-get install chromium-browser > Terminal: mccrazy@Lenovo-N22:~$ sudo add-apt-repository ppa:canonical-chromium-builds/stage [sudo] password for mccrazy:  Testing site just before upload to Ubuntu main. Things here are either broken and not ready to use, or landing in the distro anyway very soon. You shouldn't use this.  More info: https://launchpad.net/~canonical-chromium-builds/+archive/ubuntu/stage Press [ENTER] to continue or ctrl-c to cancel adding it gpg: keyring `/tmp/tmp6cfppy_r/secring.gpg' created gpg: keyring `/tmp/tmp6cfppy_r/pubring.gpg' created gpg: requesting key 5B393194 from hkp server keyserver.ubuntu.com gpg: /tmp/tmp6cfppy_r/trustdb.gpg: trustdb created gpg: key 5B393194: public key "Launchpad PPA for Canonical Chromium Build Team" imported gpg: Total number processed: 1 gpg:               imported: 1  (RSA

Moodlerooms Webservice - How to insert a user using function core_user_create_users

https://our-sandbox.mrooms.net/webservice/rest/server.php?wstoken=OUT_TOKEN_HERE&wsfunction=core_user_create_users&moodlewsrestformat=xml&users [0][username]= biboyatienza@gmail.com &users[0][password]=00.00.0000&users[0][firstname]=biboy&users[0][lastname]=atienza&users[0][email]= biboyatienza@gmail.com &users[0][city]=Manila&users[0][country]=Philippines &users[0][customfields][0][type]= customer &users[0][customfields][0][value]=Customer1 &users[0][customfields][1][type]= customeremail &users[0][customfields][1][value]= customer@email.com &users[0][customfields][2][type]= customerphone &users[0][customfields][2][value]=+6325217788