MAVERICKS

Hidden Patterns and Drivers of Academic Scholarship

Agenda

  • Gather a detailed overview of the Topic.
  • Gather the business requirements for the project.
  • Understand the scope and future of the system.

Highlights

  • The Project is going to be an extension of an already existing visualization of Scholar Plot.
  • The visualization is currently done in Javascript which limits the dynamic appearance of such Big Database.
  • The Team need to work on the Visual Representation of the data to make it more interactive and dynamic and easy to understand.
  • The main visualization should be able to clearly depict the growth of the particular Scholar over the years.
  • The project will make use of other existing third party visualization tools and APIs.
  • Visualization of Big Data is a hot topic which still in its early stages and provides a big window of opportunity to Researchers, Universities, Companies and Funding agencies.

Action Plan for the current week

  • Mr. Dinesh of University of Houston will assist in formulating the functional requirements of the project.
  • The Team will have a live video conference with Mr. Dinesh on Friday 09-02- 2016
  • Before the next class meeting, the team will research on various visualization tools to come up with an efficient way of Scholar representation.
  • The Team will also gather the Hardware and Software requirements of the Project.

Agenda

  • Meet the mentor for the first time.
  • Gather the functional and technical requirements for the project.
  • Arrange a weekly schedule to meet with the mentor.

Highlights

  • The web application will focus on growth in number of citations for a single year at a time.
  • The web application should depict the growth in citations for publications in each year in an animated movie like manner.
  • The height of value in Y axis values should increase according to the total number of citations received that year.
  • A play and pause button to control the animation.
  • The database is in JSON format.
  • The Technical Requirements will be finalized in the next meeting.
  • Mentor will provide the actual data in two weeks.
  • Mentor agreed to meet with the team every week according to Team’s interest.
  • The team can use any technology, tool or framework for the visualization.

Action Plan for the current week

  • Research on possible Javascript libraries that can satisfy the requirements in an easy, functional and flexible way.
  • The Team will start working on creating a prototype based on a tool of interest with a dummy set of values of 3-4 years.
  • Team will fix a time schedule for the weekly meeting with Mentor.

Agenda

  • Discuss and Research on various tools, APIs and packages that can perform data visualization.
  • Improve understanding of JSON.
  • Discuss on the input data format for the visualization.
  • Discuss the logic of the visualization.
  • Discuss on time slot for weekly Mentor Meetings.

Highlights

  • Javascript can be used for visualization with the addition of external packages.
  • JSON is very efficient and simple to use in extracting data.
  • The input data can contain fields for current year, Year of publication, number of citations and Publication ID.
  • The preferable time slots for mentor meeting are Monday to Wednesday 10 am to 1 pm.
  • The visualization can dynamically change graph according to the input of live data or for each data stream, a new graph can replace the previous graph.

Action Plan for the current week

  • Send Email to mentor with the preferred time slots.
  • Finalize on the tools and APIs to be used.
  • Prepare the Initial Technical report.
  • Meet again on Friday for finalizing technical requirements.

Agenda

  • Review the development till data on the Prototype.
  • Continue working on the prototype with a dummy set of JSON data.
  • Discuss the agenda for next Skype meeting with mentor.
  • The prototype increases Y values as new data comes in and removes old data points.
  • As the Y values successfully updates, the X values remain the same.

Highlights

  • The prototype is able to plot the values for the chart with live updates from the database.
  • The prototype increases Y values as new data comes in and removes old data points.
  • As the Y values successfully updates, the X values remain the same.

Action Plan for the current week

  • Ask mentor about varying X axis as per live data.
  • Continue working on the prototype with server hosted dummy data.
  • Continue working on initial technical report to include High level architecture diagram and usecase diagram.

Agenda

  • Discuss issues with prototype
  • Get corrections on initial technical requirements.

Highlights

  • Chrome developer tools can be used for debugging.
  • The Team needs to only work on the front end with the database provided by the mentor.

Action Plan for the current week

  • Work on prototype to include animation of X axis.
  • Improve UI with effects, colors and animations.
  • Research more on packages that can be helpful in UI experience.
  • Create a modified technical report.

Agenda

  • Continue working on prototype to include X axis animation and Play/Pause button
  • Formulate ideas for improving appearance of Chart.
  • Decide agenda and time for next mentor meeting.

Highlights

  • Play and Pause button works, with few bugs.
  • Hitting play after pause makes the chart to update from the beginning instead of resuming.
  • Axes should be differentiated from grid lines.
  • Improve appearance of buttons.
  • Apply smooth transition on Y axis updates.
  • In next mentor meeting, ask to mentor about reviews on current progress and get his ideas on improvising charts.

Action Plan for the current week

  • Continue working on prototype for fixing bugs.
  • Research on libraries that can provide smooth transitions.
  • Research on improving appearance by referring samples.
  • Meet with mentor on Thursday, 09-29- 2016 at 11:00am

Agenda

  • Review of current prototype.
  • Get feedback on corrected bugs in prototype.
  • Ask about UI improvement.

Highlights

  • Mentor suggested improvements on appearance of chart.
  • The prototype can now be implemented with a bigger database with different symbols.
  • Testing on different browsers can be helpful in cross platform functionality assessment.
  • A documentation can be done on various challenges faced and steps taken to resolve them.
  • The mentor will provide link for a detailed database within next week.

Action Plan for the current week

  • Add more data with different publication types.
  • Add thick lines for Axes and transparent lines for grid.
  • Include Axis labels.
  • Hovering over symbols should show URL to scholar’s Google page.
  • Parse the new database and test functionality and efficiency on different browsers.

Agenda

  • Discuss problem of updating multiple data points dynamically.
  • Continue working on prototype to include more data points.
  • Discuss agenda for mentor meeting.
  • Update axis and grid appearance.

Highlights

  • Axes lines changed to bold and grid lines made transparent.
  • Included more data points for same publication type.
  • Agenda for mentor meeting was finalized.

Action Plan for the current week

  • Talk with mentor on Wednesday, 10-05- 2016.
  • Discuss mentor’s opinion on updating data points.
  • Continue working on architecture and use case diagrams.
  • Start working on Mid review presentation.

Agenda

  • Show progress and get feedback.
  • Discuss problem of updating multiple data points dynamically.
  • Accessing each data point with ID.

Highlights

  • Change the structure of data as Paper, Year, Citations or Year, Paper, Citations.
  • Sort the years before plotting.
  • When clicking on one point with more than one publications, they should spread apart for better understanding.
  • For each year, update if there is an existing point, and plot point if there is no existing point.

Action Plan for the current week

  • Determine data format.
  • Add code for sorting data according to year.
  • Improve update function according to requirements.
  • Add more publications (30 -40) and check consistency.
  • Documentation for challenges faced and steps taken to solve them.
  • Mid Presentation draft.

Agenda

  • To discuss problem of displaying total number of citation for each paper.
  • Sorting data according to year.
  • Discuss about defining data structures for parallel update of each citation for all papers.

Highlights

  • Sort data based on publication Title and year.
  • Got suggestions for how to define pseudo data structures.

Action Plan for the current week

  • Data sorting as a part of preprocessing.
  • Citation display and parallel update of citations for all papers.

Agenda

  • Sort the data according to publication title and year of publication.
  • Work on parallel update of Data values.
  • Discuss on how to display total number of citations.

Highlights

  • Sorted data according to publication title.
  • Prototype now updates different data values in parallel.
  • To display total number of citations, a different data structure has to be defined.

Action Plan for the current week

  • Work on data structure to display total number of citations
  • Continue working on architecture and use case diagrams.
  • Continue working on Mid review presentation.

Agenda

  • Get feedback on sorted data as a part of preprocessing
  • To show up work of displaying parallel update of citations for all papers.

Highlights

  • The current stage requires some improvements in UI.
  • Dinesh will provide a new sample dataset within next week for testing.
  • Discussed about how we implemented data structure for parallel update of citations for all journals, conferences and patents.
  • Suggested below action plan for further improvement.

Action Plan for the current week

  • Work on symbols and colors to distinguish papers, journals, conferences.
  • Define flat height till 1000 for y-axis.
  • Remove comma from year values.
  • Hovering should have some content such as URL. This content needs to be defined.

Agenda

  • Improve User Interface of prototype.
  • Make updates to project Website.
  • Make changes to project Timeline.
  • Finalize mid review presentation.

Highlights

  • Colors, Heading styles and font of prototype improved.
  • Project Timeline was edited to include changes in tasks and was exported to pdf to be updated in website.
  • Mid review presentation was edited to add details of new improvements.
  • Project website was updated with Project Timeline, Mid review presentation and Team Roles.
  • Presentation structure was planned.

Action Plan for the current week

  • Meet with mentor to get feedback on updated code and improved UI.
  • Transfer JSON data to dcm server to bypass web security error in browsers.
  • Review presentation and plan for presentation on Friday, 10-21-2016.
  • Test the system with data provided by mentor.
  • Update project website with Meeting minutes and References.
  • Get ready for presentation on Friday.

Agenda

  • Ask about further improvements need to be done.
  • Request data of two more professors for testing.

Highlights

  • While creating bitbucket, keep as private.
  • Provided suggestions on what needs to be done as part of UI improvement.
  • Focus only on UI this week.
  • Mentor will provide two more databases for testing in this week.

Action Plan for the current week

  • No tooltip while animation is playing.
  • When hovering over overlapping data points, spread apart them.
  • Show detailed information about publication in tooltip.
  • Show legends.
  • Under title, show the total number of citations and h-index.

Agenda

  • Work on improving UI as per details given by mentor.
  • Update bitbucket account.
  • Discuss on how to show overlapping data points.

Highlights

  • Tool tip now shows only when paused.
  • Tooltip now shows Title and co-authors.
  • Under title, Total number of citations and H-index is shown.

Action Plan for the current week

  • Show legends in the graph.
  • Add data to tooltip.
  • Test prototype with data when mentor sends it.
  • Work on spreading apart data points if they are overlapping.
  • Talk with mentor on Wednesday.

Agenda

  • Discuss problem with data where there is no pub-citations.
  • Inform mentor that canvasJs currently has no solutions for differentiating multiple data points at a single spot.
  • Get mentor’s suggestions on displaying legends not as a chart object.
  • Get mentor’s suggestions on how to display zero as the starting point in X axis while in logarithmic scale.

Highlights

  • Mentor is contended with the problem of overlapping data points.
  • A different API or package may have solution for differentiating overlapping data points.
  • From now on, main challenges are integration and UI improvement.
  • In scholarplot, zero is shown in X axis by hardcoding the point and not as a logarithmic value.

Action Plan for the current week

  • Add a value smaller than 1 as the first value on X axis.
  • Include Download option.
  • Try to see if it is feasible to define the data as three different arrays for each type of publication, to solve the problem with legends. If not, then give the legend as an HTML object.
  • Add dynamic scaling on X axis if using decimal or use logarithmic.
  • Refer high charts to see if they have solutions for overlapping data points.
  • Think individually on the design of Backend data base for efficient formatting and accessing of data.
  • Deploy data in UHCL dcm server and send link to mentor.

Agenda

  • Get mentor’s help/review on current database designs.
  • Get agenda for the rest of the weeks.
  • Update mentor with current improvements made last week.

Highlights

  • Database with JSON values are acceptable.
  • The design should be easy to update and maintainable in the future.
  • Mentor reviewed updates on Scaling and accepted them.

Action Plan for the current week

  • Experiment with dummy data for SQL table.
  • Change the Scale selection button to a Toggle button.
  • Create a Help documentation for the users.

    Agenda

    • Install MYSQL workbench.
    • Create sample schema with provided code.
    • Create agenda for next mentor meeting.

    Highlights

    • JSON datatype is not recognized.
    • UI improved to bring in font changes and color schemas.
    • Learned about creating SQL tables with JSON values.
    • Researched on updating and maintaining JSON fields.
    • Created database design.

    Action Plan for the current week

  • Ask to mentor about JSON datatype.
  • Try with different versions of MYSQL.
  • Create demo tables.
  • Update Bitbucket
  • Remind professor about hosting data in dcm server.

    Agenda

    • Ask mentor on how to add JSON datatype.
    • Get information about PHP scripting for data retrieval.

    Highlights

    • Use pub_year and citation intead of author column.
    • For ID, combine google_id and publication_id.
    • Try insertion and retrieving of values.
    • Insertion can be done manually or with a PHP script.
    • Mentor provided links on PHP scripts to be used.

    Action Plan for the current week

  • Update the mentor within next two days with changes made.
  • Start on creating Documentation. Get ideas from professor in Today’s meeting.
  • Implement the changes prescribed by mentor.
  • Research on how to retrieve portion of JSON values.
  • Update the project website.
  • The demo links in project website need to be updated with proper databases.
  • Send link to mentor after professor hosts files in dcm server.

    Agenda

    • Try JSON datatype creation with updated MYSQL version.
    • Extract publication_id and google_id and put together as publication_id.
    • Try Insert and Retrieve.
    • Insertion can be done manually or with PHP script.

    Highlights

    • JSON datatype works with version 6.3
    • Table created as per schema provided by Mentor.
    • Portion of JSON values have to be extracted for publication_id and google_id.

    Action Plan for the current week

    • Research on how to retrieve portion of JSON values.
    • Try Insertion and Retrieving with demo data.
    • Instead of author, give publication_year and citations.

    Agenda

    • Report progress to Mentor
    • Ask mentor for PHP script to format SQL data in JSON format.
    • Ask mentor about installing and running Apache server with XAMP.

    Highlights

    • Mentor gave ideas on how to create PHP script.
    • For XAMPP, use a different unused port in the system.
    • In PHP, use for loop in array to format pubcitations.
    • Make corrections to SQL table.

    Action Plan for the current week

  • Create PHP script.
  • Continue working on PHP array creation.
  • Start creating presentation and final report draft.
  • Submit draft on Tuesday.
  • Gather the ideas for report and presentation from Google drive.

    Agenda

    • Continue working on PHP script.
    • Correct errors in connection.
    • Look at netstat in XAMPP for finding used ports.
    • Create raw data for presentation and report.

    Highlights

    • Presentation draft has to be submitted on Tuesday.
    • All data which after midterm are uploaded in Google Drive.
    • Task needs to be divided for report creation and presentation creation.
    • Give equal importance to all tasks.

    Action Plan for the current week

  • Submit Presentation draft.
  • Continue working on Report and Presentation.
  • Copy valid slides from Mid review presentation.
  • Update HLL design of System and Use case diagram.
  • Gather information about formatting into JSON with PHP.
  • Report progress to mentor by Monday.

    Agenda

    • Report progress to Mentor
    • Ask mentor for help in formatting JSON.
    • Ask mentor for co-authors column.

    Highlights

    • Data is formatted to suit the JSON format.
    • All redundant array values are corrected.
    • Continue without co-authors column.
    • Got action plan for current week.

    Action Plan for the current week

  • Integrate PHP with JS code.
  • Work on formatting of PHP file.
  • Update the Project website
  • Work on Presentation and Report.
  • Do Presentation demo.

    Agenda

    • Continue working on PHP script.
    • Continue working on Final Presentation and Report.
    • Gather ideas for final presentation.

    Highlights

    • Title array redundancy is solved.
    • Pub citations are now displayed.
    • Continue working on presentation and focus on report after 7th.
    • PHP needs to be hosted in server.

    Action Plan for the current week

  • Host PHP file.
  • Test integration.
  • Update Final Presentation.
  • Create presentation according to guidelines given by professor.
  • Practice presentation demo.