'Below are some ideas that have been listed out as possibilities for helping us with the debug process as we go forward with testing on live sites. We need to figure out which of these are worth implementing given the dev time left this year.
The ideas below are meant to help us in solving problems on live sites more quickly. Either by being able to replicate the issue or by preemptively preparing.
Ideas:
- Detailed error report when the crawler crashes and dies.
-- Is this implemented as a try-catch around anytime we interact with something that prints out a small report with the state num, action, and element? Some errors are not thrown in here though.
- "Skipping" functionality that allows the crawler to jump straight to the state that it died on in a previous run.
-- Any chance we could pickle our web access or controller after each state? How slow is object serialization? This could be made a config mode?
- Use an intern to scan for various programming approaches and determine ways to counteract them early on?
-- Unsure of if an intern would have a good enough grasp on our tool and on variety of programming approaches? How to bridge that gap to make output useful.
- Try running on websites that cause overwhelming problems (E.g., cnn where we can't even currently load the page).
@jstanley-mitre @jphiggins-mitre @dchud I would appreciate feedback on the ideas above as well as my supposed implementation on some of them. Please add onto the list if you have come up with anything else.
'Below are some ideas that have been listed out as possibilities for helping us with the debug process as we go forward with testing on live sites. We need to figure out which of these are worth implementing given the dev time left this year.
The ideas below are meant to help us in solving problems on live sites more quickly. Either by being able to replicate the issue or by preemptively preparing.
Ideas:
-- Is this implemented as a try-catch around anytime we interact with something that prints out a small report with the state num, action, and element? Some errors are not thrown in here though.
-- Any chance we could pickle our web access or controller after each state? How slow is object serialization? This could be made a config mode?
-- Unsure of if an intern would have a good enough grasp on our tool and on variety of programming approaches? How to bridge that gap to make output useful.
@jstanley-mitre @jphiggins-mitre @dchud I would appreciate feedback on the ideas above as well as my supposed implementation on some of them. Please add onto the list if you have come up with anything else.