[tbb-commits] [Git][tpo/applications/tor-browser-spec][main] 3 commits: Create bugzilla2gitlab script for ESR resolved issue audits
morgan (@morgan)
git at gitlab.torproject.org
Tue Oct 22 18:50:31 UTC 2024
morgan pushed to branch main at The Tor Project / Applications / tor-browser-spec
Commits:
1472857c by Richard Pospesel at 2024-06-27T04:18:44+00:00
Create bugzilla2gitlab script for ESR resolved issue audits
- fetches all resolved bugs for a firefox release
- outputs gitlab markdown for each entry which:
- displays bugzilla issue number, title
- links to bugzilla issue
- shows a button which when clicked populates a review issue prepopulated with:
- bugzilla information
- appropriate gitlab labels
- links to parent audit issue
- provides checklist for engineers to mark blocks as triaged
- - - - -
aaf00ad7 by Morgan at 2024-10-22T18:49:55+00:00
updated code_audit.sh script to handle .mjs js files and some minor tweaks
- - - - -
d3418425 by Morgan at 2024-10-22T18:50:15+00:00
FF116-FF128 Audits
- - - - -
15 changed files:
- + audits/FF116_AUDIT
- + audits/FF117_AUDIT
- + audits/FF118_AUDIT
- + audits/FF119_AUDIT
- + audits/FF120_AUDIT
- + audits/FF121_AUDIT
- + audits/FF122_AUDIT
- + audits/FF123_AUDIT
- + audits/FF124_AUDIT
- + audits/FF125_AUDIT
- + audits/FF126_AUDIT
- + audits/FF127_AUDIT
- + audits/FF128_AUDIT
- + audits/bugzilla2gitlab.sh
- audits/code_audit.sh
Changes:
=====================================
audits/FF116_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: `9c13862f3e084cec78650fa01450f6d18aec1530` ( `FIREFOX_ESR_115_BASE` )
+- End: `ff486626d0de0e7f34d65ef000c657080ddf564d` ( `FIREFOX_116_0_3_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF117_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: ff486626d0de0e7f34d65ef000c657080ddf564d ( `FIREFOX_116_0_3_RELEASE` )
+- End: 6f3830e39c76ae6d0ab19b4f9289d434d424cbe3 ( `FIREFOX_117_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF118_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at 6f3830e39c76ae6d0ab19b4f9289d434d424cbe3 ( `FIREFOX_117_0_RELEASE` )
+- End: tor-browser at a928b6c0612a2690852fa3b5d13efc2a80868a90 ( `FIREFOX_118_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF119_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at a928b6c0612a2690852fa3b5d13efc2a80868a90 ( `FIREFOX_118_0_RELEASE` )
+- End: tor-browser at 7ab3cc0103090dd7bfa02e072a529b9fc784ab4e ( `FIREFOX_119_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF120_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at 7ab3cc0103090dd7bfa02e072a529b9fc784ab4e ( `FIREFOX_119_0_RELEASE` )
+- End: tor-browser at dedee7a8c6cbabc80294733634360f6fbeeeadc0 ( `FIREFOX_120_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF121_AUDIT
=====================================
@@ -0,0 +1,28 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at dedee7a8c6cbabc80294733634360f6fbeeeadc0 ( `FIREFOX_120_0_RELEASE` )
+- End: tor-browser at a32b8662993085139ac91212a297123b632fc1c0 ( `FIREFOX_121_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+#### 1add9d4c13a6493e670d01b38f4eb839c53bf1ba
+- Mozilla 1815739: Support using Firefox as default PDF reader on Android
+- https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43159
+- Review Result: SAFE
+
+#### a6562d5849a78c58340bb3d9b975f1208db4401d
+- Mozilla 1852340: Implement a new "report broken site" feature for desktop Firefox
+- https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43160
+- Review Result: SAFE
=====================================
audits/FF122_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at a32b8662993085139ac91212a297123b632fc1c0 ( `FIREFOX_121_0_RELEASE` )
+- End: tor-browser at 7e38fabb90748649da04ed45a2f80d68423362d9 ( `FIREFOX_122_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF123_AUDIT
=====================================
@@ -0,0 +1,30 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at 7e38fabb90748649da04ed45a2f80d68423362d9 ( `FIREFOX_122_0_RELEASE` )
+- End: tor-browser at f8704c84a751716bad093b9bdc482db53fe5b3ea ( `FIREFOX_123_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
+
+#### 14797b7fa8c5df0332ba5d422803dbcdf548c056
+#### eb73825495faf333a4fe812316ac38e138f5bf8d
+#### 818788a96a700c6d44a17ab1e932de96cc45eac6
+#### c0aa048b3918e367e9fd84442695f1fbb2087f30
+- https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43161
+- Mozilla 1852900: Pass HTTPS requests to native resolver thread
+- Mozilla 1852902: Allow nsINativeDNSResolverOverride to override native HTTPS records
+- https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/43161
+- Review Result: SAFE
=====================================
audits/FF124_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at f8704c84a751716bad093b9bdc482db53fe5b3ea ( `FIREFOX_123_0_RELEASE` )
+- End: tor-browser at eb063e98ca624ff7d430a9b9aa356381f49e2e5a ( `FIREFOX_124_0_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF125_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at eb063e98ca624ff7d430a9b9aa356381f49e2e5a ( `FIREFOX_124_0_RELEASE` )
+- End: tor-browser at 59577ab1445892568bafb39124e5757a307177f2 ( `FIREFOX_125_0_BUILD1` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF126_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: 59577ab1445892568bafb39124e5757a307177f2 ( `FIREFOX_125_0_BUILD1` )
+- End: 5889d9823cc5975561827262efeb24464360402c ( `FIREFOX_126_0_BUILD1` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF127_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: 5889d9823cc5975561827262efeb24464360402c ( `FIREFOX_126_0_BUILD1` )
+- End: e480e7382673f60d2f8590e7018d291b52e982b0 ( `FIREFOX_127_0b1_RELEASE` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/FF128_AUDIT
=====================================
@@ -0,0 +1,20 @@
+# General
+
+The audit begins at the commit hash where the previous audit ended. Use code_audit.sh for creating the diff and highlighting potentially problematic code. The audit is scoped to a specific language (currently C/C++, Rust, Java/Kotlin, and Javascript).
+
+The output includes the entire patch where the new problematic code was introduced. Search for `XXX MATCH XXX` to find the next potential violation.
+
+`code_audit.sh` contains the list of known problematic APIs. New usage of these functions are documented and analyzed in this audit.
+
+## Firefox: https://github.com/mozilla/gecko-dev.git
+
+- Start: tor-browser at e480e7382673f60d2f8590e7018d291b52e982b0 ( `FIREFOX_127_0b1_RELEASE` )
+- End: tor-browser at 9352d2be309c27f0e93471e2bb3352d7cfb76052 ( `FIREFOX_128_0b1_BUILD1` )
+
+### Languages:
+- [x] java
+- [x] cpp
+- [x] js
+- [x] rust
+
+Nothing of interest (using `code_audit.sh`)
=====================================
audits/bugzilla2gitlab.sh
=====================================
@@ -0,0 +1,122 @@
+#!/usr/bin/env bash
+
+echoerr() { echo "$@" 1>&2; }
+
+if [ "$#" -lt 3 ]; then
+ echoerr "Usage: $0 firefox-version gitlab-audit-issue-number reviewers... > output.md"
+ exit 1
+fi
+
+# Check pre-conditions
+check_exists() {
+ local cmd=$1
+ if ! which ${cmd} > /dev/null ; then
+ echoerr "missing ${cmd} dependency"
+ exit 1
+ fi
+}
+
+check_exists wget
+check_exists jq
+check_exists sed
+check_exists perl
+
+# assign arguments to named variables
+firefox_version=$1
+audit_issue=$2
+reviewers="${@:3}"
+
+# check valid esr version
+if ! [[ "${firefox_version}" =~ ^[1-9][0-9]{2}$ ]]; then
+ echoerr "invalid Firefox version (probably)"
+ exit 1
+fi
+
+# check valid issue number
+if ! [[ "${audit_issue}" =~ ^[1-9][0-9]{4}$ ]]; then
+ echoerr "invalid gitlab audit issue number (probably)"
+ exit 1
+fi
+
+# download bug list
+json=/tmp/${firefox_version}.json
+bugzilla_query="https://bugzilla.mozilla.org/buglist.cgi?j_top=OR&f1=target_milestone&o3=equals&v3=Firefox%20${firefox_version}o1=equals&resolution=FIXED&o2=anyexact&query_format=advanced&f3=target_milestone&f2=cf_status_firefox${firefox_version}&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&v1=mozilla128&v2=fixed%2Cverified&limit=0"
+# you can get this from the 'REST' link at the bottom of the prevoius bugzilla query ^^;
+bugzilla_json_query="https://bugzilla.mozilla.org/rest/bug?include_fields=id,summary,status&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&f1=target_milestone&f2=cf_status_firefox${firefox_version}&f3=target_milestone&j_top=OR&limit=0&o1=equals&o2=anyexact&o3=equals&resolution=FIXED&v1=mozilla128&v2=fixed%2Cverified&v3=Firefox%20${firefox_version}"
+
+wget "${bugzilla_json_query}" -O ${json}
+
+echo "### [Bugzilla Query](${bugzilla_query})"
+echo ""
+
+issue_count=$(jq '.bugs | length' ${json})
+counter=0
+jq '.bugs | sort_by(.id)[] | "\(.id)|\(.summary)"' ${json} | while IFS='|' read -r id summary; do
+
+ # indexing
+ counter=$((counter + 1))
+
+ from=$counter
+ through=$((counter + 499))
+ if ((to > issue_count)); then
+ to=$issue_count
+ fi
+
+ # break up into sections or else gitlab falls over
+ if ((counter % 500 == 1)); then
+ echo "<details>"
+ echo " <summary>Resolved Firefox ${firefox_version} Bugzilla Issues ${from} through ${through}</summary>"
+ echo ""
+ fi
+
+ # bugzilla info
+ id="${id:1}"
+ summary="${summary:0:-1}"
+ [[ ${#summary} -gt 90 ]] && summary_short="${summary:0:87}..." || summary_short="${summary}"
+
+ # we need to escape printed strings for markdown
+ md_escape() {
+ local input="$1"
+ # jesus I'm sorry
+ echo "${input}" | sed 's/[][\\`*_{}<>()#+-\.~]/\\&/g'
+ }
+
+ md_summary=$(md_escape "${summary}")
+ md_summary_short=$(md_escape "$summary_short")
+
+ # we need to urlencode the strings used in the new issue link
+ url_encode() {
+ local input="$1"
+ echo "${input}" | perl -MURI::Escape -wlne 'print uri_escape $_'
+ }
+
+ # parent issue
+ bugzilla_url="https://bugzilla.mozilla.org/show_bug.cgi?id=${id}"
+ # review issue title
+ new_issue_title=$(url_encode "Review Mozilla ${id}: ${summary_short}")
+ # review issue description
+ new_issue_description=$(url_encode "### Bugzilla: ${bugzilla_url}")%0A$(url_encode "/label ~\"14.0 stable\" ~FF128-esr ~Next")%0A$(url_encode "/relate tpo/applications/tor-browser-spec#${audit_issue}")%0A%0A$(url_encode "<!-- briefly describe why this issue needs further review -->")%0A
+ # url which create's new issue with title and description pre-populated
+ new_issue_url="../../../../tor-browser/-/issues/new?issue[title]=${new_issue_title}&issue[description]=${new_issue_description}"
+
+ # em-space
+ em=" "
+ counter_string=$(printf "%04i" ${counter})
+
+ echo "- **${counter_string}**${em}<kbd>[Create Issue](${new_issue_url})</kbd>${em}[**${id}**: ${md_summary}](${bugzilla_url})"
+
+
+ if ((counter % 500 == 0 )) || (( counter == issue_count )); then
+ # checklist of engineers that have triaged this block
+ echo "</details>"
+ echo
+ echo "**Triaged by:**"
+ for reviewer in $reviewers; do
+ echo "- [ ] **${reviewer}**"
+ done
+ echo
+ elif ((counter % 25 == 0 )); then
+ # add a hrule every 25 to break things up visually
+ echo "---"
+ fi
+done
=====================================
audits/code_audit.sh
=====================================
@@ -138,7 +138,7 @@ case "${SCOPE}" in
initialize_rust_symbols
;;
"js" )
- EXT="js jsm"
+ EXT="js jsm mjs"
initialize_js_symbols
;;
* )
@@ -172,9 +172,9 @@ rm -f "${REPORT_FILE}"
# of said commit
# Flashing Color constants
-export GREP_COLOR="05;37;41"
+export GREP_COLORS="mt=05;37;41"
-for COMMIT in $(git rev-list --ancestry-path $OLD~..$NEW); do
+for COMMIT in $(git log --format="%H" $NEW ^$OLD); do
TEMP_DIFF="$(mktemp)"
echo "Diffing $COMMIT..."
View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-spec/-/compare/10d1b43ec94a0538027d6ecb668c91c321c38bcd...d341842552ea14cac7bfc106cd3a433925f18841
--
View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-spec/-/compare/10d1b43ec94a0538027d6ecb668c91c321c38bcd...d341842552ea14cac7bfc106cd3a433925f18841
You're receiving this email because of your account on gitlab.torproject.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tbb-commits/attachments/20241022/a2594ca0/attachment-0001.htm>
More information about the tbb-commits
mailing list