google_test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Before you can use the Google API, you must do the following:\n",
    "\n",
    "Follow instructions at:\n",
    "    https://programmablesearchengine.google.com/controlpanel/all?optin=true\n",
    "\n",
    "Select Add.\n",
    "\n",
    "On next page:\n",
    "•\tName your engine\n",
    "•\tSearch entire web\n",
    "•\tI’m not a robot\n",
    "•\tCreate\n",
    "\n",
    "On next page (Your new search engine has been created), select “Go to legacy Control Panel” at the top.\n",
    "Next, select Continue.\n",
    "\n",
    "On next page:\n",
    "    Select your search engine\n",
    "    Copy your search engine ID\n",
    "    Copy your public URL\n",
    "\n",
    "Next, go to https://developers.google.com/custom-search/v1/introduction and \n",
    "select ‘Get a Key’. In the ‘Enable Custom Search API’, enter a new project \n",
    "name. Next, select ‘Show Key’ and copy your API key: \n",
    "\n",
    "Next, fill in your API_KEY and SEARCH_ENGINE_ID below.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\work\\git\\tech_scout\\.venv\\lib\\site-packages\\urllib3\\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.76.225.15'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----------\n",
      "Title: Sean Warnick\n",
      "Link: https://science.byu.edu/directory/sean-warnick\n",
      "Snippet: Sean Warnick received the B.S.E. degree from Arizona State University in 1993, and the S.M. and Ph.D. degrees from the Massachusetts Institute of Technology ...\n",
      "Pagemap: {'cse_thumbnail': [{'src': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQjEjCvkCgdp2lIDlTftTAuxJxr8FHlKejiFgSu9dPLblT9t5SvYEFP_IE', 'width': '225', 'height': '225'}], 'metatags': [{'og:image': 'https://brightspotcdn.byu.edu/dims4/default/c6657d4/2147483647/strip/true/crop/300x300+0+0/resize/1200x1200!/quality/90/?url=https%3A%2F%2Fbrigham-young-brightspot.s3.amazonaws.com%2F50%2Faf%2F83b0cc02535c86cb0356b105b3fb%2Fpersonphoto-cgi-1.jpeg', 'og:image:width': '1200', 'og:type': 'profile', 'twitter:card': 'summary_large_image', 'twitter:title': 'Sean Warnick', 'og:site_name': 'The College of Physical and Mathematical Sciences (CPMS)', 'og:image:url': 'https://brightspotcdn.byu.edu/dims4/default/c6657d4/2147483647/strip/true/crop/300x300+0+0/resize/1200x1200!/quality/90/?url=https%3A%2F%2Fbrigham-young-brightspot.s3.amazonaws.com%2F50%2Faf%2F83b0cc02535c86cb0356b105b3fb%2Fpersonphoto-cgi-1.jpeg', 'profile:username': 'Sean Warnick', 'og:title': 'Sean Warnick', 'og:image:height': '1200', 'og:image:type': 'image', 'twitter:image': 'https://brightspotcdn.byu.edu/dims4/default/f24efba/2147483647/strip/true/crop/300x169+0+66/resize/1200x675!/quality/90/?url=https%3A%2F%2Fbrigham-young-brightspot.s3.amazonaws.com%2F50%2Faf%2F83b0cc02535c86cb0356b105b3fb%2Fpersonphoto-cgi-1.jpeg', 'profile:last_name': 'Warnick', 'viewport': 'width=device-width, initial-scale=1, maximum-scale=5', 'brightspot.contentid': '00000172-5d75-df29-a773-7d773e280010', 'profile:first_name': 'Sean C', 'og:url': 'https://science.byu.edu/directory/sean-warnick'}], 'webpage': [{'affiliation': 'Computational Science and Control Research, Computer Science', 'name': 'Sean Warnick'}], 'cse_image': [{'src': 'https://brightspotcdn.byu.edu/dims4/default/c6657d4/2147483647/strip/true/crop/300x300+0+0/resize/1200x1200!/quality/90/?url=https%3A%2F%2Fbrigham-young-brightspot.s3.amazonaws.com%2F50%2Faf%2F83b0cc02535c86cb0356b105b3fb%2Fpersonphoto-cgi-1.jpeg'}], 'article': [{'articlebody': 'Biography Sean Warnick received the B.S.E. degree from Arizona State University in 1993, and the S.M. and Ph.D. degrees from the Massachusetts Institute of Technology in 1995 and 2003, respectively...', 'description': 'Sean Warnick received the B.S.E. degree from Arizona State University in 1993, and the S.M. and Ph.D. degrees from the Massachusetts Institute of Technology in 1995 and 2003, respectively....'}]}\n",
      "-----------\n",
      "Title: Sean Warnick at Brigham Young University - RateMyProfessors.com\n",
      "Link: https://www.ratemyprofessors.com/professor?tid=712633\n",
      "Snippet: Sean Warnick is a professor in the Computer Science department at Brigham Young University - see what their students are saying about them or leave a rating ...\n",
      "Pagemap: {'metatags': [{'og:image': 'https://www.ratemyprofessors.com/static/media/meta.9c8a7c2b.png', 'theme-color': '#000000', 'og:type': 'website', 'twitter:card': 'summary_large_image', 'viewport': 'width=device-width, initial-scale=1', 'og:title': 'Sean Warnick at Brigham Young University - RateMyProfessors.com', 'og:url': 'https://www.ratemyprofessors.com/professor?tid=712633', 'og:description': 'Check out the ratings and reviews for Sean Warnick from Brigham Young University'}], 'cse_image': [{'src': 'https://www.ratemyprofessors.com/static/media/meta.9c8a7c2b.png'}]}\n",
      "-----------\n",
      "Title: ‪Sean Warnick‬ - ‪Google Scholar‬\n",
      "Link: https://scholar.google.com/citations?user=k7WBrxoAAAAJ&hl=en\n",
      "Snippet: Professor of Computer Science, Brigham Young University - ‪‪Cited by 1555‬‬ - ‪Mathematical Systems Theory‬ - ‪Feedback Control‬ - ‪Network Systems‬ ...\n",
      "Pagemap: {'hcard': [{'fn': 'Sean Warnick', 'title': 'Professor of Computer Science, Brigham Young University'}], 'cse_thumbnail': [{'src': 'https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcRf1_iC3eHHuK9y1mN86sbepnGVZDrRAZJkU7uJCQm8luErFI0L2PQNdKpB', 'width': '225', 'height': '225'}], 'person': [{'role': 'Professor of Computer Science, Brigham Young University', 'org': 'cs.byu.edu'}, {'jobtitle': 'Professor of Computer Science, Brigham Young University', 'worksfor': 'cs.byu.edu', 'name': 'Sean Warnick'}], 'metatags': [{'referrer': 'origin-when-cross-origin', 'og:image': 'https://scholar.google.com/citations/images/avatar_scholar_256.png', 'og:type': 'website', 'twitter:card': 'summary', 'viewport': 'width=device-width,initial-scale=1,minimum-scale=1,maximum-scale=2', 'og:title': 'Sean Warnick', 'og:description': '\\u202aProfessor of Computer Science, Brigham Young University\\u202c - \\u202a\\u202aCited by 1,555\\u202c\\u202c - \\u202aMathematical Systems Theory\\u202c - \\u202aFeedback Control\\u202c - \\u202aNetwork Systems\\u202c - \\u202aEmerging Applications\\u202c', 'format-detection': 'telephone=no'}], 'itemlist': [{'itemlistelement': 'Necessary and sufficient conditions for dynamical structure reconstruction of LTI networksJ Gonçalves, S WarnickIEEE Transactions on Automatic Control 53 (7), 1670-1674, 20082412008'}], 'cse_image': [{'src': 'https://scholar.google.com/citations/images/avatar_scholar_256.png'}]}\n",
      "-----------\n",
      "Title: Sean Warnick\n",
      "Link: https://idealabs.byu.edu/people/sean-warnick/\n",
      "Snippet: Sean Warnick received his Ph.D. and S.M. degrees from the Massachusetts Institute of Technology in 1995 and 2003, respectively, and his B.S.E. from Arizona ...\n",
      "Pagemap: {'cse_thumbnail': [{'src': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTbSB-AZLCDLK4WQL63-Uwoi6iFLSi7BcKUgtqSJBrS2NMLSKVW2Omax4g', 'width': '194', 'height': '259'}], 'metatags': [{'viewport': 'width=device-width, initial-scale=1'}], 'cse_image': [{'src': 'https://idealabs.byu.edu/images/people/SeanWarnick_hu03a7683027f9e4d97e4937d680065fb3_1843012_0x500_resize_q75_box.jpg'}]}\n",
      "-----------\n",
      "Title: Sean Warnick\n",
      "Link: https://cs.byu.edu/department/directory/faculty-directory/sean-warnick/\n",
      "Snippet: Sean was named the Distinguished Visiting Professor by the National Security Agency three years in a row, 2008-2010, for his work with their Summer Program for ...\n",
      "Pagemap: {'cse_thumbnail': [{'src': 'https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcRPBg2JdwkxgbvoEZT_wZccrV4tm4aK2LIjlj8nCOMPeOv8gm2ju_Y1zpY', 'width': '225', 'height': '225'}], 'metatags': [{'viewport': 'width=device-width, initial-scale=1.0'}], 'cse_image': [{'src': 'https://cs.byu.edu/media/images/Warnick_Sean_01.1a6dff02.fill-720x720-c100.jpg'}]}\n",
      "-----------\n",
      "Title: Charles Warnick Obituary (2022) - Legacy Remembers\n",
      "Link: https://www.legacy.com/us/obituaries/name/charles-warnick-obituary?id=35953153\n",
      "Snippet: Jul 23, 2022 ... Charles Peter Warnick, known as Chuck among his friends, passed away on ... with a chemistry minor, from BYU in Provo, Utah in 1957.\n",
      "Pagemap: {'cse_thumbnail': [{'src': 'https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQjzIxKRX9-o57Q1WvQEeQ35mHqlVjwudKyh5Pu4wZ3PY1WOxvSAec7VUkl', 'width': '310', 'height': '163'}], 'Event': [{'image': 'https://cache.legacy.net/legacy/images/cobrands/legacyadn/photos/a075f972-48f0-45f0-b629-182a013aac24.pngx?w=266&h=305&option=3', 'endDate': '2022-07-28T10:30:00Z', 'name': 'Visitation for Charles Warnick', 'description': 'Funeral and Services for Charles Warnick', 'startDate': '2022-07-28T09:30:00Z'}, {'image': 'https://cache.legacy.net/legacy/images/cobrands/legacyadn/photos/a075f972-48f0-45f0-b629-182a013aac24.pngx?w=266&h=305&option=3', 'name': 'Service for Charles Warnick', 'description': 'Funeral and Services for Charles Warnick', 'startDate': '2022-07-28T11:00:00Z'}], 'metatags': [{'og:image': 'https://cache.legacy.net/legacy/images/cobrands/legacyadn/photos/a075f972-48f0-45f0-b629-182a013aac24.pngx?w=600&h=315', 'og:type': 'ARTICLE', 'og:image:width': '600', 'og:site_name': 'Legacy.com', 'og:title': 'Charles Warnick Obituary (2022) - Legacy Remembers', 'og:image:height': '315', 'fb:pages': '54460902620', 'og:description': \"View Charles Warnick's obituary, send flowers and sign the guestbook.\", 'og:image:secure_url': 'https://cache.legacy.net/legacy/images/cobrands/legacyadn/photos/a075f972-48f0-45f0-b629-182a013aac24.pngx?w=600&h=315', 'fb:app_id': '162166070516109', 'viewport': 'width=device-width, initial-scale=1, maximum-scale=1', 'fb:admins': '100001241414690', 'og:url': 'https://www.legacy.com/us/obituaries/name/charles-warnick-obituary?id=35953153'}], 'cse_image': [{'src': 'https://cache.legacy.net/legacy/images/cobrands/legacyadn/photos/a075f972-48f0-45f0-b629-182a013aac24.pngx?w=600&h=315'}]}\n",
      "-----------\n",
      "Title: BYU Computer Science Department\n",
      "Link: https://cs.byu.edu/news/\n",
      "Snippet: ... included club booths, Frisbee, J-Dawgs, and a raffle! Stay tuned for future events on the CS Instagram page: @byucomputsci ... Advisor: Sean Warnick.\n",
      "Pagemap: {'cse_thumbnail': [{'src': 'https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcR4ZNGLt7BpR0YX52gaCFa7W_D8mLS_ZWnKLL-3Xkzli8F_YblUEbQnux0', 'width': '225', 'height': '225'}], 'metatags': [{'viewport': 'width=device-width, initial-scale=1.0'}], 'cse_image': [{'src': 'https://cs.byu.edu/media/images/research_post.width-250.png'}]}\n",
      "-----------\n",
      "Title: Students study bees to create waggle dance translator - The Daily ...\n",
      "Link: https://universe.byu.edu/2021/10/11/students-study-bees-to-create-waggle-dance-translator/\n",
      "Snippet: Oct 11, 2021 ... Computer science students at BYU are studying this performance to create a ... but with the help of his advisor Sean Warnick, that interest ...\n",
      "Pagemap: {'hcard': [{'fn': \"Ally O'Rullian\"}], 'cse_thumbnail': [{'src': 'https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcTW3m6sMwh0vBf-ZiZH5-jzuQpS6mFdxwS6evRKvBPVdtcJVMf5U7JlIxk', 'width': '270', 'height': '187'}], 'metatags': [{'og:image': 'https://universe.byu.edu/wp-content/uploads/2021/10/Screenshot-44-1.jpg', 'og:type': 'article', 'article:published_time': '2021-10-11T08:24:00+00:00', 'og:image:width': '1433', 'twitter:card': 'summary_large_image', 'og:site_name': 'The Daily Universe', 'author': \"Ally O'Rullian\", 'og:title': 'Students study bees to create waggle dance translator - The Daily Universe', 'og:image:height': '992', 'twitter:label1': 'Written by', 'twitter:label2': 'Est. reading time', 'og:image:type': 'image/jpeg', 'og:description': 'The tango, the cha-cha and the waltz all pale in comparison to the waggle dance performed by honeybees that tells other bees the direction and distance to find plants. Computer science students at BYU are studying this performance to create a monitor that can translate their movements in real time.', 'fb:pages': '19738093244', 'article:publisher': 'https://www.facebook.com/UniverseBYU/', 'twitter:data1': \"Ally O'Rullian\", 'twitter:data2': '2 minutes', 'article:modified_time': '2021-10-18T17:24:40+00:00', 'viewport': 'width=device-width, initial-scale=1.0', 'og:locale': 'en_US', 'og:url': 'https://universe.byu.edu/2021/10/11/students-study-bees-to-create-waggle-dance-translator/'}], 'cse_image': [{'src': 'https://universe.byu.edu/wp-content/uploads/2021/10/Screenshot-44-1.jpg'}]}\n",
      "-----------\n",
      "Title: Dynamics of Self-Enforcing Cooperation in Competitive Environments\n",
      "Link: http://jur.byu.edu/?p=1127\n",
      "Snippet: Jun 18, 2013 ... Nathan Woodbury and Dr. Sean Warnick, Department of Computer Science. Introduction. My original purpose in this project was to “explore the ...\n",
      "Pagemap: {'person': [{'name': 'admin', 'url': 'admin'}], 'metatags': [{'viewport': 'width=device-width, initial-scale=1'}], 'creativework': [{'text': 'Nathan Woodbury and Dr. Sean Warnick, Department of Computer Science Introduction My original purpose in this project was to “explore the nature of self-enforcing policies that induce participati...', 'headline': 'Dynamics of Self-Enforcing Cooperation in Competitive Environments', 'datepublished': '2013-06-18T17:41:38+00:00'}]}\n",
      "-----------\n",
      "Title: Integrated Scheduling and Control in Discrete-time with Dynamic ...\n",
      "Link: https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=3127&context=facpub\n",
      "Snippet: Apr 10, 2018 ... Logan D.R. Beal, Damon Petersen, David Grimsman, Sean Warnick, ... aDepartment of Chemical Engineering, Brigham Young University, Provo, ...\n",
      "Pagemap: {'cse_thumbnail': [{'src': 'https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcT11b1B0MT0MTCQE81t_dJsb_5YS3h8YfGBHB5ujbu2GpPUTv2eTlqiHTk', 'width': '255', 'height': '197'}], 'metatags': [{'moddate': \"D:20220708083212-07'00'\", 'creationdate': \"D:20220708083212-07'00'\", 'creator': 'Appligent AppendPDF Pro 6.3', 'author': 'Logan Beal, Damon Petersen, David R. Grimsman, Sean Warnick, and John Hedengren', 'producer': 'Prince 12.5 (www.princexml.com)', 'appligent': 'AppendPDF Pro 6.3 Linux 64 bit Aug 30 2019 Library 15.0.4', 'title': 'Integrated Scheduling and Control in Discrete-time with Dynamic Parameters and Constraints'}], 'cse_image': [{'src': 'x-raw-image:///8a99e21c298d0ec5638f137f8f5c107858fd063806d57f6283c9151513cf7aa3'}]}\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "# get the API KEY here: https://developers.google.com/custom-search/v1/overview\n",
    "API_KEY = \"<you API KEY here>\"\n",
    "\n",
    "# get your Search Engine ID on your CSE control panel\n",
    "SEARCH_ENGINE_ID = \"<your Search Engine ID here>\"\n",
    "\n",
    "# the search query you want\n",
    "query = \"sean warnick byu\"\n",
    "# using the first page\n",
    "page = 1\n",
    "# constructing the URL\n",
    "# doc: https://developers.google.com/custom-search/v1/using_rest\n",
    "# calculating start, (page=2) => (start=11), (page=3) => (start=21)\n",
    "start = (page - 1) * 10 + 1\n",
    "url = f\"https://www.googleapis.com/customsearch/v1?key={API_KEY}&cx={SEARCH_ENGINE_ID}&q={query}&start={start}\"\n",
    "\n",
    "# make the API request\n",
    "data = requests.get(url, verify=False).json()\n",
    "\n",
    "# get the result items\n",
    "search_items = data.get(\"items\")\n",
    "\n",
    "# iterate over 10 results found\n",
    "for i, search_item in enumerate(search_items, start=1):\n",
    "    print('-----------')\n",
    "    # Use the following to see other available keys\n",
    "    #print(f'{search_item}, type: {type(search_item)}')\n",
    "    print(f'Title: {search_item[\"title\"]}')\n",
    "    print(f'Link: {search_item[\"link\"]}')\n",
    "    print(f'Snippet: {search_item[\"snippet\"]}')\n",
    "    print(f'Pagemap: {search_item[\"pagemap\"]}')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.6 ('.venv': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "b9217cfbd83c61c810f65951f71541107733f7b399dbf4d5390730f82cee703e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}