Conversation
* if no search index exists in mongo, one is created * if the fields of the existing search index in mongo do not match the current model, the existing search index is updated
* the academic calendars are now uploaded with an additional url field containing the box url for the calendar * bugfix: fixed past selector so that the links are extracted correctly and completely Co-authored-by: SuchDominion shijiand2022@gmail.com
|
The bug with the past selector was that it was simply a duplicate of the future selector. Now, the past selector selects all the links in both div elements for the past section. This fix resolves the issue that we were having with duplicate links in the csv in the scraping phase which led to duplicates in the json in the parsing stage. Now, all the calendars should be there. |
TyHil
left a comment
TyHil
left a comment
There was a problem hiding this comment.
I like the use of the CSV! I think the intent of including the link it for it to be listed for people to visit. For that reason, can you use the link to the Box page (like https://utdallas.app.box.com/s/hwavyx0pbojyb3ymn2kfvl0pwkwkkto6) instead of the static link (this one: https://utdallas.box.com/shared/static/%s.pdf)?
The changes in uploader/uploader.go don't seem relevant to the issue being solved but I may be missing something
|
The changes in the uploader is related to the search index used for fuzzy searching discount programs. It is here because that PR is not merged to develop yet, so it’s fine since it needs to be merged before this. Also there’s a conflict to resolve :(( |
|
Oh yeah, I caused the conflict so just resolved it |
3 participants
Fixes: #158