├── .github └── workflows │ └── cla.yml ├── README.md ├── approved ├── 001-keyed-expansions.md └── 002-retweets.md ├── principles.md ├── proposal-template.md ├── rejected ├── 000-delete-rules-endpoint.md └── 000-relation_info_on_user_lookup.md └── superseded └── 000-consistent-id-type.md /.github/workflows/cla.yml: -------------------------------------------------------------------------------- 1 | name: "Developer Agreement" 2 | on: 3 | issue_comment: 4 | types: [created] 5 | pull_request_target: 6 | types: [opened,closed,synchronize] 7 | 8 | jobs: 9 | agreement: 10 | runs-on: ubuntu-latest 11 | steps: 12 | - name: "CLA Assistant" 13 | if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read and agree to the Twitter Developer Agreement and Policy') || github.event_name == 'pull_request_target' 14 | uses: cla-assistant/github-action@ba066dbae3769e2ce93ec8cfc4fdc51b9db628ba #v2.1.3-beta 15 | env: 16 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 17 | PERSONAL_ACCESS_TOKEN : ${{ secrets.CLA_PAT }} 18 | with: 19 | remote-organization-name: twitter 20 | remote-repository-name: .github-private 21 | path-to-signatures: 'developer-agreement/signatures.json' 22 | custom-pr-sign-comment: 'I have read and agree to the Twitter Developer Agreement and Policy' 23 | custom-notsigned-prcomment: '
Thank you for your submission, we really appreciate it. We ask that you agree to the [Twitter Developer Agreement and Policy](https://developer.twitter.com/en/developer-terms/agreement-and-policy) before we can accept your proposal. You can agree by posting a comment with the following text:
' 24 | custom-allsigned-prcomment: 'All contributors have signed the Developer Agreement. If the commit check is not passing, a maintainer must go the Checks tab of this PR and rerun the GitHub Action.' 25 | lock-pullrequest-aftermerge: false 26 | branch: 'main' 27 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Twitter API Open Evolution 2 | 3 | This repository allows you to submit proposals to change aspects of the Twitter public API. You can read more about this process below, and in [the announcement post](https://twittercommunity.com/t/introducing-twitter-open-evolution/158770) on the Twitter Developer Community forums. 4 | 5 | ## Submitting a proposal 6 | 7 | Anyone can submit a proposal for consideration by Twitter. Proposals must conform to a defined template, and must respect the [general principles of the Twitter API platform](principles.md). 8 | 9 | Before submitting a proposal, we strongly encourage you to start a discussion by [opening an issue](https://github.com/twitterdev/open-evolution/issues) on GitHub, and to seek consensus and input from the broader developer community. 10 | 11 | ## Workflow 12 | 13 | 1. Fork this repository. 14 | 2. Make a copy of [proposal-template.md](proposal-template.md), and rename it so that the title describes the proposal (it can have the same title as your proposal). Use a format like `000-your-proposal-name.md`, where `your-proposal-name` is a descriptive name (e.g. the proposal title), keeping `000` for the numeric part. 15 | 3. Fill the proposal template by following the instructions within. 16 | 4. Submit a Pull Request. As soon as a review manager is available, they will contact the authors to discuss the proposal. 17 | 5. Once a review is scheduled, be ready and prepared to discuss your proposal with the review manager. Expect the process to last no more than 15 working days. 18 | 6. At the end of the review process, a decision will be made and the proposal can be approved or rejected. The review manager can also request you to submit a revision; in this case, the proposal is sent back and another review will be scheduled. 19 | 20 | ## Important things to note 21 | 22 | - When a submission is accepted, it means that Twitter _agrees in principle_ with the proposed changes. This does _not_ mean that the proposal will be immediately implemented. 23 | - While all decisions are final and may not be appealed, rejected proposals can be revised and resubmitted, as long as they respect the [general principles of the Twitter API platform](principles.md). 24 | - It is the responsibility of the authors to check that the proposal is not a duplicate of a previously accepted or rejected proposal. 25 | - Submissions must refer to the current version of the API; previous versions may remain supported by the platform, but proposals for changes to these versions will be rejected (however, a proposal may be resubmitted against the current version, assuming that it does not duplicate a previously accepted or rejected proposal). 26 | - Submissions must adhere to the [general principles of the Twitter API platform](principles.md). Submissions that do not meet these requirements may be rejected without a review. In this case, a core team member (possibly the member who flagged the rejection candidate) will assign themselves as a review manager to execute the rejection, or to ask the author to submit a revision. 27 | - A review can terminate early when a proposal is withdrawn from consideration, or when a decision can be made within the review period. 28 | 29 | ## Statuses 30 | 31 | At any given time, a proposal can be in one of the following statuses: 32 | 33 | - **Open.** The initial status of a proposal. 34 | - **Reviewable.** The proposal has been accepted by a review manager. The review manager proceeds to schedule a review period and communicate it to the authors. 35 | - **Reviewing.** The proposal is in active review. 36 | - **Withdrawn.** The authors withdrew the proposal. 37 | - **Rejected.** The proposal has been reviewed and rejected. 38 | - **Approved.** The proposal has been reviewed and approved. 39 | - **Revision needed.** The proposal has been reviewed but it needs to be sent back to the authors for changes before it can be rejected or approved. 40 | - **Superseded.** A newer or more relevant proposal or implementation makes this proposal obsolete or no longer applicable. This status applies to open or approved proposals. 41 | 42 | Remember: When a submission is accepted, it means that Twitter _agrees in principle_ with the proposed changes. This does _not_ mean that the proposal will be implemented. This can happen for example when further privacy and security review highlight a risk, or when there is a change in roadmap or priorities, or in case of an already scheduled release of related functionality that supersedes a proposal. 43 | 44 | ## What to expect 45 | 46 | - You can submit a proposal at any time for review. Submissions can be in the Open state for a while, but we will review all proposals. 47 | - Once a proposal is in review, you can expect the review to take 15 business days. Sometimes, company and country holidays may get in the way; if that happens, we will add more time to the review process. 48 | - You can expect infrequent but timely communication. That's what we'll be expecting from you, too. 49 | - You can expect feedback that describes the motivation for a rejection. 50 | -------------------------------------------------------------------------------- /approved/001-keyed-expansions.md: -------------------------------------------------------------------------------- 1 | # Return keyed dictionaries for expanded objects 2 | 3 | - Proposal: (link to Github PR related to this proposal) 4 | - Discussion: (link to Github issue related to this proposal) 5 | - Authors: 6 | - Daniele ([Twitter](https://twitter.com/i_am_daniele), [Github](https://github.com/iamdaniele)) 7 | 8 | - Review Manager: Brent Halsey [Github](https://github.com/bhalsey) 9 | - Status: Approved 10 | 11 | ## Introduction 12 | 13 | Change the arrays in the `includes` payload into keyed dictionaries for easier lookup. 14 | 15 | ## Motivation 16 | 17 | The current design of the Twitter API contains a main `data` object and an `includes` object that gets returned when the user requests an expansion: 18 | 19 | ```bash 20 | $ twurl '/2/tweets?ids=20,21&expansions=author_id' 21 | ``` 22 | 23 | ```js 24 | { 25 | "data": [ 26 | { 27 | "author_id": "12", 28 | "id": "20", 29 | "text": "just setting up my twttr" 30 | }, 31 | { 32 | "author_id": "13", 33 | "id": "21", 34 | "text": "just setting up my twttr" 35 | } 36 | ], 37 | "includes": { 38 | "users": [ 39 | { 40 | "id": "12", 41 | "name": "jack", 42 | "username": "jack" 43 | }, 44 | { 45 | "id": "13", 46 | "name": "Biz Stone", 47 | "username": "biz" 48 | } 49 | ] 50 | } 51 | } 52 | ``` 53 | 54 | There are two problems with the current implementation: 55 | 56 | 1. There is no explicit reference or mapping to what entity needs to be looked up in `includes`. In this example `author_id` matches to `includes.users.id`, but that is not immediately explicit to a user unless the documentation is consulted. 57 | 1. `includes` returns zero or more arrays. The user needs to cycle through the content of each array to lookup the information requested, which is clearly introduces problem a suboptimal computational resource usage, especially over a high number of requests. 58 | 59 | This is particularly relevant when requesting expansion and looking up multiple results at the same time; this is the case for Search or Follower graph, where the user is requesting enough data to display tweets on mobile devices with enough data to satisfy Twitter's display requirement policy. 60 | 61 | ```bash 62 | $ twurl '/2/tweets/search/recent?query=cats has:images&expansions=author_id,attachments.media_keys&max_results=100' 63 | ``` 64 | 65 | ```js 66 | { 67 | "data": [ 68 | // up to 100 tweets 69 | ], 70 | "includes": { 71 | "users": [ 72 | {"id": "1"}, 73 | {"id": "2"}, 74 | {"id": "3"}, 75 | {"id": "4"}, 76 | // up to 100 elements 77 | ], 78 | "media": [ 79 | {"media_key": "1_1234676"}, 80 | {"media_key": "1_1234677"}, 81 | {"media_key": "1_1234678"}, 82 | {"media_key": "1_1234679"}, 83 | // up to 400 elements on a 100-results page 84 | ] 85 | }, 86 | "meta": { 87 | "newest_id": "…", 88 | "oldest_id": "…", 89 | "result_count": 100, 90 | "next_token": "…" 91 | } 92 | } 93 | ``` 94 | 95 | In this example, users who wish to reconcile `data` with `includes` will need to implement three theoretically O(n) lookups. An incomplete implementation will look like this (using Swift as a reference): 96 | 97 | ```swift 98 | extension TweetLookupResponse { 99 | func user(of tweet: Tweet) -> User? { 100 | return includes?.users?.first { $0.id == tweet.authorId } 101 | } 102 | 103 | func media(tweet: Tweet) -> [Media]? { 104 | if let mediaKeys = tweet.attachments?.mediaKeys, 105 | let media = self.includes?.media { 106 | return media.filter { (item) -> Bool in 107 | return mediaKeys.contains(item.mediaKey) 108 | } 109 | } 110 | return nil 111 | } 112 | ``` 113 | 114 | And so on for the other `includes` (tweets and poll data). 115 | 116 | While the implementation is reasonably simple, it introduces two main issues: 117 | 118 | - It's not clear if the API guarantees that a user or media will be in `includes`. A lengthy lookup can turn to no results, with unforeseen circumstances. 119 | - We're adding 2 * O(n) complexity, which is unneeded if we mapped or keyed the content of `includes` 120 | 121 | This complexity can be avoided by making each `includes` object a dictionary, and keying these dictionaries by their `id`. This does not introduce a radically new usage pattern, and it does not not change the way `includes` are requested. 122 | 123 | ## Proposed solution 124 | 125 | Make the objects in `include` dictionaries instead of arrays, and key their objects by their `id`: 126 | 127 | ```bash 128 | $ twurl '/2/tweets?ids=20,21&expansions=author_id' 129 | ``` 130 | 131 | ```js 132 | { 133 | "data": [ 134 | { 135 | "author_id": "12", 136 | "id": "20", 137 | "text": "just setting up my twttr" 138 | }, 139 | { 140 | "author_id": "13", 141 | "id": "21", 142 | "text": "just setting up my twttr" 143 | } 144 | ], 145 | "includes": { 146 | "users": { 147 | "12": { 148 | "id": "12", 149 | "name": "jack", 150 | "username": "jack" 151 | }, 152 | "13": { 153 | "id": "13", 154 | "name": "Biz Stone", 155 | "username": "biz" 156 | } 157 | } 158 | } 159 | } 160 | ``` 161 | 162 | For a multi result lookup: 163 | ```bash 164 | $ twurl '/2/tweets/search/recent?query=cats has:images&expansions=author_id,attachments.media_keys&max_results=100' 165 | ``` 166 | 167 | ```js 168 | { 169 | "data": [ 170 | // up to 100 tweets per page 171 | ], 172 | "includes": { 173 | "users": { 174 | "1": {"id": "1"}, 175 | "2": {"id": "2"}, 176 | "3": {"id": "3"}, 177 | "4": {"id": "4"}, 178 | // up to 100 elements per page 179 | }, 180 | "media": [ 181 | "1_1234676": {"media_key": "1_1234676", "url": "…"}, 182 | "1_1234677": {"media_key": "1_1234677", "url": "…"}, 183 | "1_1234678": {"media_key": "1_1234678", "url": "…"}, 184 | "1_1234679": {"media_key": "1_1234679", "url": "…"}, 185 | // up to 400 elements on a 100-results page 186 | ] 187 | }, 188 | "meta": { 189 | "newest_id": "…", 190 | "oldest_id": "…", 191 | "result_count": 100, 192 | "next_token": "…" 193 | } 194 | } 195 | ``` 196 | 197 | There is no change in the request. The response only changes in the `includes` payload, only when requested. 198 | 199 | Because the dictionary is keyed by the `id` of the requested element, this change improves the developer experience by removing the need for developers to implement unnecessary search (access to a dictionary is direct). Our previous Swift will not need to extend the data model, and can be indexed directly as: 200 | 201 | 202 | ```swift 203 | 204 | let fullTweet = data.map { 205 | let user = includes?.users[$0.author_id]? 206 | let media = $0.attachments?.mediaKeys.map { (key) in 207 | return includes?.media[key]? 208 | } 209 | } 210 | ``` 211 | 212 | And so on for other `includes`. 213 | 214 | This solution is also cleaner compared to the current implementation because it reduces the amount of code developers have to deal with, and makes it explicit what has been expanded by just counting keys instead of counting array elements. 215 | 216 | In terms of both data transfer and data protection, this solution does not offer significant advantages and it does not introduces significant detriment. 217 | 218 | ## Detailed design 219 | 220 | The payload structure will change from an JSON array to a JSON dictionary: 221 | 222 | ```js 223 | "includes": { // dictionary (same as current) 224 | "media": { // dictionary (change from current) 225 | "id": {} // media object (same as current) 226 | }, 227 | "tweets": { // dictionary (change from current) 228 | "id": {} // tweet object (same as current) 229 | }, 230 | "users": { // dictionary (change from current) 231 | "id": {} // user object (same as current) 232 | }, 233 | "polls": { // dictionary (change from current) 234 | "id": {} // poll object (same as current) 235 | }, 236 | "places": { // dictionary (change from current) 237 | "id": {} // place object (same as current) 238 | }, 239 | } 240 | ``` 241 | 242 | In OpenAPI, this could be implemented by coding an `additionalProperties` payload, keyed to be a string: 243 | 244 | ```yaml 245 | Expansions: 246 | type: object 247 | properties: 248 | users: 249 | type: object 250 | additionalProperties: 251 | $ref: '#/components/schemas/User' 252 | minItems: 1 253 | tweets: 254 | type: object 255 | additionalProperties: 256 | $ref: '#/components/schemas/Tweet' 257 | minItems: 1 258 | places: 259 | type: object 260 | additionalProperties: 261 | $ref: '#/components/schemas/Place' 262 | minItems: 1 263 | media: 264 | type: object 265 | additionalProperties: 266 | $ref: '#/components/schemas/Media' 267 | minItems: 1 268 | polls: 269 | type: object 270 | additionalProperties: 271 | $ref: '#/components/schemas/Poll' 272 | minItems: 1 273 | ``` 274 | 275 | ## Compatibility, breaking changes and migrations 276 | 277 | This proposal will introduce a breaking change in the `includes` payload. 278 | 279 | Developers who are not interested in `includes`, such as those who do not request `expansions`, can safely ignore the change. The remaining developers can easily migrate in the following ways. 280 | 281 | 1. Ignore dictionary keys and iterate over each element of the dictionary as if it was an array. This works best with loosely typed languages. For example, in JavaScript, developers can maintain compatibility by using `Object.values(includes.users)` which will discard the keys in `includes.users` and convert the dictionary into an array with no further action required. 282 | 2. Implement direct object access and removing the now unnecessary lookup code. 283 | 284 | Because this is a breaking change, if implemented today, developers may notice errors; this applies only if the developer is requesting `expansions`, and only if the `includes` payload is returned. Some SDKs may also require to be adapted in order to remove unnecessary lookup code (if implemented) and to ensure compatibility for the new mapping. An initial analysis of Twitter SDKs does not indicate the presence of lookup code to match an expansion to the main response body; this suggests the migration would be reasonably easy. 285 | 286 | ## Alternatives considered 287 | 288 | There are some alternatives to this proposed approach. 289 | 290 | ### Do nothing 291 | 292 | Doing nothing will keep the need to perform lookups on each request payload; developers may end up creating additional structures to optimize lookups in case of high usage, which adds complexity for those developers who use the API on a large scale. For every developer, regardless of their expertise, it's an unnecessary obstacle that can be removed, so they can focus on less foundational issues. In the longer term, this can be a hurdle for all developers who use the API in large volumes. 293 | 294 | ### Expand objects in-place 295 | 296 | The basic principle of an expansion is to populate the ID of an entity with its full object. For this reason, it can be envisioned that the `includes` payload can be removed and not returned; instead, expansions could populate in place of its `id`: 297 | 298 | ```bash 299 | $ twurl '/2/tweets?ids=20,21&expansions=author_id' 300 | ``` 301 | 302 | ```js 303 | { 304 | "data": [ 305 | { 306 | "author_id": { 307 | "id": "12", 308 | "name": "jack", 309 | "username": "jack" 310 | }, 311 | "id": "20", 312 | "text": "just setting up my twttr" 313 | }, 314 | { 315 | "author_id": { 316 | "id": "13", 317 | "name": "Biz Stone", 318 | "username": "biz" 319 | }, 320 | "id": "21", 321 | "text": "just setting up my twttr" 322 | } 323 | ] 324 | } 325 | ``` 326 | 327 | This solution is similar to the [expansion design adopted by Stripe](https://stripe.com/docs/expand); it has the advantage of removing the need to lookup additional information in the `includes` object, but it has the following disadvantages: 328 | 329 | 1. It has type safety implication, as the data model would need to accept either a string or an object 330 | 1. If an API has results belonging to the same entity (e.g. more than one tweet belonging to a user), the response is forced to populate the same information twice, leading to redundant information and increased response size 331 | 1. In this specific example, `author_id` looks semantically wrong (the field is `author_id`, but the API returns an object). 332 | -------------------------------------------------------------------------------- /approved/002-retweets.md: -------------------------------------------------------------------------------- 1 | # Retweets 2 | 3 | - Proposal: https://github.com/twitterdev/open-evolution/pull/4 4 | - Discussion: https://github.com/twitterdev/open-evolution/pull/4 5 | - Authors: 6 | - Igor Brigadir (@igorbrigadir, @igorbrigadir) 7 | 8 | - Review Manager: TBD (this will be filled by the review manager) 9 | - Status: Open 10 | 11 | ## Introduction 12 | 13 | Add a new retweets endpoint. 14 | 15 | ## Motivation 16 | 17 | The current v2 API https://developer.twitter.com/en/docs/twitter-api/tweets/retweets/introduction only replaces the v1.1 https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/api-reference/get-statuses-retweeters-ids endpoint, which only gives you the *users* that have retweeted a tweet, not the *retweet* itself. 18 | 19 | The retweet itself is important to get the full tweet metadata, especially `created_at` and ID of the retweet. The timestamps are important for modeling retweet spread and the other metadata that goes along with a retweet (user, public metrics from the `referenced_tweet`) are also important to maintain counts and a sense of how things change over time with retweets. 20 | 21 | ## Proposed solution 22 | 23 | Add a new endpoint `/2/tweets/:id/retweets` that gives you the actual retweets themselves instead of the users. This will be a replacement for https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/api-reference/get-statuses-retweets-id 24 | 25 | ## Detailed design 26 | 27 | The proposed new endpoint for retweets will closely mirror https://developer.twitter.com/en/docs/twitter-api/tweets/timelines/api-reference/get-users-id-mentions having the same query parameters like `since_id`, `until_id`, fields and expansions etc, except the `id` path parameter is of a tweet not a user. 28 | 29 | For example: 30 | 31 | `GET /2/tweets/1441054496931541004/retweets` 32 | 33 | Using https://twitter.com/TwitterDev/status/1451609611455242241 as an example, calling `GET /2/tweets/1441054496931541004/retweets` (usual fields and expansions parameters ommitted) should give you: 34 | 35 | 36 | ``` 37 | { 38 | "data": [ 39 | { 40 | "id": "1452022021651517441", 41 | "referenced_tweets": [ 42 | { 43 | "type": "retweeted", 44 | "id": "1451609611455242241" 45 | } 46 | ], 47 | "text": "RT @TwitterDev: Join us on Friday October 29th at 2 pm ET for our next live stream on academic research using the #TwitterAPI. We are excit…", 48 | "author_id": "1375017222813310981", 49 | "conversation_id": "1452022021651517441" 50 | }, 51 | { 52 | "id": "1451852904764149761", 53 | "referenced_tweets": [ 54 | { 55 | "type": "retweeted", 56 | "id": "1451609611455242241" 57 | } 58 | ], 59 | "text": "RT @TwitterDev: Join us on Friday October 29th at 2 pm ET for our next live stream on academic research using the #TwitterAPI. We are excit…", 60 | "author_id": "2906851412", 61 | "conversation_id": "1451852904764149761" 62 | }, 63 | { 64 | "id": "1451644319098720257", 65 | "referenced_tweets": [ 66 | { 67 | "type": "retweeted", 68 | "id": "1451609611455242241" 69 | } 70 | ], 71 | "text": "RT @TwitterDev: Join us on Friday October 29th at 2 pm ET for our next live stream on academic research using the #TwitterAPI. We are excit…", 72 | "author_id": "744265436", 73 | "conversation_id": "1451644319098720257" 74 | }, 75 | { 76 | "id": "1451625579640344576", 77 | "referenced_tweets": [ 78 | { 79 | "type": "retweeted", 80 | "id": "1451609611455242241" 81 | } 82 | ], 83 | "text": "RT @TwitterDev: Join us on Friday October 29th at 2 pm ET for our next live stream on academic research using the #TwitterAPI. We are excit…", 84 | "author_id": "1153115735192932353", 85 | "conversation_id": "1451625579640344576" 86 | } 87 | ], 88 | "includes": { 89 | "tweets": [ 90 | { 91 | "id": "1451609611455242241", 92 | "attachments": { 93 | "media_keys": [ 94 | "3_1451606862583926784" 95 | ] 96 | }, 97 | "text": "Join us on Friday October 29th at 2 pm ET for our next live stream on academic research using the #TwitterAPI. We are excited to host @dfreelon and Andrew Crist from @unc_citap to share how they use the Twitter API for research studies. We will be live on https://t.co/GrtBOXh5Y1 https://t.co/URtpGcU3ZK", 98 | "conversation_id": "1451609611455242241", 99 | "author_id": "2244994945" 100 | } 101 | ], 102 | "users": [ 103 | { 104 | "name": "LaiAvery", 105 | "username": "taishen8808", 106 | "id": "1375017222813310981" 107 | }, 108 | { 109 | "name": "Twitter Dev", 110 | "pinned_tweet_id": "1441054496931541004", 111 | "username": "TwitterDev", 112 | "id": "2244994945" 113 | }, 114 | { 115 | "name": "nov it's jack birthday⚡", 116 | "pinned_tweet_id": "1174968413254406144", 117 | "username": "novitsjackbdy", 118 | "id": "2906851412" 119 | }, 120 | { 121 | "name": "Newspaper & Online News Division of AEJMC", 122 | "username": "aejmc_nond", 123 | "id": "744265436" 124 | }, 125 | { 126 | "name": "Center for Information, Technology, & Public Life", 127 | "username": "unc_citap", 128 | "id": "1153115735192932353" 129 | } 130 | ] 131 | }, 132 | "meta": { 133 | "newest_id": "1452022021651517441", 134 | "oldest_id": "1451625579640344576", 135 | "result_count": 4, 136 | "next_token": "b26v89c19zqg8o3fpdv67sb97sn3b0wzcwvnn8it32a2l" 137 | } 138 | } 139 | ``` 140 | 141 | ## Compatibility, breaking changes and migrations 142 | 143 | As this is a new endpoint, there are no breaking changes. 144 | 145 | ## Alternatives considered 146 | 147 | In v1.1 API the eaquivalent is https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/api-reference/get-statuses-retweets-id or the Account Activity API which requires a persistent webhook to receive retweets as the happen, and cannot fetch them retrospectively. Another alternative is the `retweets_of_status_id:` operator in PowerTrack. (This operator is not available in v2 filtered stream). Just like Account Activity API, it only returns real time results. This is also available via Historical PowerTrack, but it is only available to enterprise users and managed customers. 148 | 149 | Currently the only way to do this is using v2 API Search is with `retweets_of:user` which is not adequate as it gives you all retweets of all tweets by the user, not a specific tweet. 150 | -------------------------------------------------------------------------------- /principles.md: -------------------------------------------------------------------------------- 1 | # General principles of the Twitter API platform 2 | 3 | Before submitting a proposal, it’s important to understand some fundamental principles of our API platform. 4 | 5 | ## Privacy and security 6 | 7 | Honoring and protecting the privacy of users on Twitter is a fundamental aspect of the API platform. New proposals should respect this principle and, if possible, set an even higher standard for developers. 8 | 9 | ## Consistency 10 | 11 | Endpoints that are expected to return the same objects (e.g. Tweets) are also expected to behave in the same way. For example, developers should expect a Tweet lookup endpoint and a Tweet search endpoint to accept the same fields and expansions, and to return the same payload. The search endpoint may provide additional fields (e.g. to support pagination), but the request and response structure should be very similar, if not identical. 12 | 13 | ## Versioning 14 | 15 | A new API version will be issued if the API design requires a breaking change. A breaking change is a change that will require developers to significantly change their current implementations. 16 | 17 | Once a new version is released, the previous version is supported for 12 months, while the now-new current version is supported until the next breaking change. Developers should expect new API versions no more frequently than once every 12 months. 18 | 19 | New features or additions will be considered for the current version of the API only. Previous versions remain supported for use for 12 months, and once a new version is released, the Open Evolution process will expect proposals to be submitted against this version. 20 | 21 | ## Authentication 22 | 23 | Most endpoints will require authentication, and the authentication method will be OAuth 1.0a or OAuth 2.0. We encourage submitting proposals about scopes and improvements to data granularity in the context of the OAuth 2.0 authentication method. Because we are very strict on protecting the trust our developers and users put in our service, it's unlikely that we will consider different authentication methods. We take data protection and user privacy very seriously. When submitting a proposal for this aspect of the API, we strongly recommend describing the implications on data security, and expect these proposals to encounter increased scrutiny. 24 | 25 | ## Rate limiting and activity quotas 26 | 27 | There are many ways to implement metered access to data; rate limiting and activity are just two examples. When discussing limiting and quotas, focus on the general architecture that can be applied in a coherent way across endpoints. It's possible that certain endpoints may require an additional level of access (e.g. a search endpoint may require both a rate limit and an activity quota), so if a proposal is covering these aspects, ensure to cover the circumstances where more than one access factor should be considered. 28 | 29 | Discussions about increasing or decreasing rate limits are generally considered outside the scope of the API design. 30 | -------------------------------------------------------------------------------- /proposal-template.md: -------------------------------------------------------------------------------- 1 | # Proposal title 2 | 3 | - Proposal: (link to GitHub PR related to this proposal) 4 | - Discussion: (link to GitHub issue related to this proposal) 5 | - Authors: 6 | - Author 1 (Twitter, GitHub) 7 | - Author 2 (Twitter, GitHub) 8 | 9 | - Review Manager: TBD (this will be filled by the review manager) 10 | - Status: Open 11 | 12 | ## Introduction 13 | 14 | Briefly describe what the proposal is about. This introduction is a short, one-paragraph vision statement for the proposal. 15 | 16 | ## Motivation 17 | 18 | What problem or problems is this proposal looking to address? 19 | 20 | For changes or modifications to the current API design, provide one or more examples of how a similar solution can be achieved within the current design. Practical examples are an effective way to visualize and reproduce (if possible and needed) the problem you’re trying to address with this proposal. 21 | 22 | If you are introducing new functionality, explain in detail what are the advantages and benefits compared to the current design. This section should answer the following questions: 23 | 24 | - Does this proposed solution introduce new usage patterns? What learning curve should developers expect and why? 25 | - Are there similar API designs (e.g. RFCs, peer platforms) implementing a similar solution? 26 | 27 | ## Proposed solution 28 | 29 | What is the intended solution or result? If the proposal is introducing new functionality, provide a detailed description and add examples as needed. For example, if the proposal aims to change the payload structure, provide an example request and response payload. If the proposal seeks to change the route structure of request fields or headers, provide a comprehensive list in order to describe the widest possible range of circumstances the proposal will apply to. 30 | 31 | This section should answer these questions: 32 | 33 | - How would this solution look like from the perspective of the developer? 34 | - How is this proposed solution improving the developer experience? 35 | - How is this proposed solution cleaner than the current solution? 36 | - How is this proposed solution safer than the current solution, in terms of data protection? 37 | - How is this proposed solution more efficient in terms of data requested and retrieved? 38 | 39 | ## Detailed design 40 | 41 | How would your proposal change the API design? This section should be used as the technical representation of your proposal. In this section, you detail how the grammar and ergonomics of the API should be changed. For example, if you are introducing a change in the route structure, you should explain how the route structure should be applied to each entity requested. If you are planning to add a field, expansion or payload section, explain how the behavior will be coherent across similar requests. 42 | 43 | This section should contain all the necessary information with a level of clarity that can allow the Twitter team to implement this feature. 44 | 45 | ## Compatibility, breaking changes and migrations 46 | 47 | Will this proposal require developers to change their current implementation? Your proposal should articulate how developers may implement the new feature. Some changes will require developers to change their implementation, and that is considered a breaking change. If your proposal changes the behavior of any currently implemented feature of the API design, you should clearly state that. 48 | 49 | In any case, this section should always answer these questions: 50 | 51 | - How will this proposal work with existing developer implementations? 52 | - If we implemented this proposal today, what apps will require a change, and what apps will break without this change? 53 | - How can developers discover and quickly implement this proposed design? 54 | - How will SDK or client libraries maintainer need to implement or support this feature? 55 | 56 | ## Alternatives considered 57 | 58 | Describe what other alternatives exist, or what other solutions you considered. For each alternative, your proposal should describe how this solution improves the Twitter API platform. There may be cases where certain solutions may offer a wider range of benefits than your proposal, and that’s okay. This can be the case of a solution that’s technically superior, but that may take more time and effort than what you’re proposing. In that case, explain why how your proposed solution can be a reasonable tradeoff. 59 | 60 | This section should answer these questions: 61 | 62 | - What happens if we do nothing? 63 | - What alternatives did you consider? What are the benefits and tradeoffs of your proposed solution compared to this alternative? -------------------------------------------------------------------------------- /rejected/000-delete-rules-endpoint.md: -------------------------------------------------------------------------------- 1 | # Separate endpoint for deleting filtered stream rules 2 | 3 | - Proposal: [#7](https://github.com/twitterdev/open-evolution/pull/7) 4 | - Discussion: (link to Github issue related to this proposal) 5 | - Authors: 6 | - Shubham Parihar ([Twitter](https://twitter.com/iShiibi), [Github](https://github.com/iShibi)) 7 | 8 | - Review Manager: mmerritt 9 | - Status: rejected 10 | 11 | ## Introduction 12 | 13 | Add a separate `DELETE` endpoint for deleting filtered stream rules. 14 | 15 | ## Motivation 16 | 17 | Currently, there is a single endpoint for adding and deleting filtered stream rules ([POST /2/tweets/search/stream/rules](https://developer.twitter.com/en/docs/twitter-api/tweets/filtered-stream/api-reference/post-tweets-search-stream-rules)) 18 | 19 | There are two major problems with this endpoint: 20 | 21 | 1) The endpoint is a `POST` request. It represents adding a rule correctly but the endpoint can also be used to delete a rule, which isn't a correct implementation of the spec. Deleting a resource should be done using a `DELETE` request. 22 | 23 | 2) Keeping the spec aside, the endpoint throws an error if `add` and `delete` fields are used together in the JSON body of a request to this endpoint. This raises the question that why even merge these two actions into a single endpoint when they cannot be used together at a time. 24 | 25 | 26 | ## Proposed solution 27 | 28 | Make `(POST /2/tweets/search/stream/rules)` endpoint to only be used for adding new rules and introduce a new `(DELETE /2/tweets/search/stream/rules)` endpoint for deleting the currently active rules. 29 | 30 | This will change the `body` of the requests to not have `add` and `delete` fields as the action will get inferred from the HTTP verb. 31 | 32 | This change will improve the developer experience by removing the caveats related to the current endpoint. One of them is the `add` and `delete` fields in the JSON body being mutually exclusive. 33 | 34 | ## Detailed design 35 | 36 | The requested new `DELETE` endpoint should look something like this: 37 | 38 | ```bash 39 | curl -X DELETE 'https://api.twitter.com/2/tweets/search/stream/rules' 40 | -H "Content-type: application/json" 41 | -H "Authorization: Bearer $BEARER_TOKEN" -d 42 | '{ 43 | "ids": [ 44 | "1165037377523306498", 45 | "1165037377523306499" 46 | ] 47 | 48 | // or "values" array if one wants to delete using values of rules 49 | }' 50 | ``` 51 | 52 | Making the current `POST` endpoint become: 53 | 54 | ```bash 55 | curl -X POST 'https://api.twitter.com/2/tweets/search/stream/rules' 56 | -H "Content-type: application/json" 57 | -H "Authorization: Bearer $BEARER_TOKEN" -d 58 | '{ 59 | "rules": [ 60 | {"value": "cat has:media", "tag": "cats with media"}, 61 | {"value": "cat has:media -grumpy", "tag": "happy cats with media"} 62 | ] 63 | }' 64 | ``` 65 | 66 | ## Compatibility, breaking changes and migrations 67 | 68 | This is a breaking change as the `body` of the request is getting changed. The old `add` and `delete` fields will be no more. `ids`, `values`, and the new `rules` fields will become top-level fields of the JSON body. 69 | 70 | ## Alternatives considered 71 | 72 | None 73 | -------------------------------------------------------------------------------- /rejected/000-relation_info_on_user_lookup.md: -------------------------------------------------------------------------------- 1 | # Relation information between an authenticated user and the result of users/lookup 2 | 3 | - Proposal: (link to Github PR related to this proposal) 4 | - Discussion: [(link to Github issue related to this proposal)](https://github.com/twitterdev/open-evolution/issues/5) 5 | - Author: 6 | - Redouane Bali ([@RedouaneBali](https://twitter.com/RedouaneBali), [Redouane59](https://github.com/redouane59)) 7 | 8 | - Review Manager: Daniele ([Twitter](https://twitter.com/i_am_daniele), [Github](https://github.com/iamdaniele)) 9 | - Status: Rejected 10 | 11 | ## Introduction 12 | 13 | When using the [user lookup endpoint](https://developer.twitter.com/en/docs/twitter-api/v1/accounts-and-users/follow-search-get-users/api-reference/get-users-lookup), we can't know if the returned user is following us or not. 14 | 15 | ## Motivation 16 | 17 | Using a [bot](https://twitter.com/RedTheBot_) , I would like to keep some features only for the accounts following it. But today, to know if someone interacting with it is following the bot account or not is really hard. 18 | I have to keep a list of all my followers ids locally and update it regularly. The cost of these calls are really heavy. 19 | 20 | ## Proposed solution 21 | 22 | To me, the payload of the user lookup endpoint should contain informations about the relation between the authenticated user and the other user. 23 | 24 | ## Detailed design 25 | 26 | Adding two new fields in the API response of users/lookup endpoint, i.e : 27 | 28 | ``` 29 | { 30 | "followed": false, 31 | "following": true 32 | } 33 | ``` 34 | 35 | ## Compatibility, breaking changes and migrations 36 | 37 | These two additional fields will have to be defined in libraries. 38 | 39 | ## Alternatives considered 40 | 41 | Only alternative I see would be to increase the API calls limit for get followers ids & get followings isd endpoints. 42 | Currently only 15 calls are permitted so to retrieve 30K followers, it can last up to 30min. -------------------------------------------------------------------------------- /superseded/000-consistent-id-type.md: -------------------------------------------------------------------------------- 1 | # Proposal title 2 | 3 | - Proposal: https://github.com/twitterdev/open-evolution/pull/8 4 | - Discussion: (link to Github issue related to this proposal) 5 | - Authors: 6 | - Max Qian (@NitDef, @rMaxiQp) 7 | 8 | - Review Manager: @iamdaniele 9 | - Status: Superseded (platform change to return ID as strings has been released in production) 10 | 11 | ## Introduction 12 | 13 | Making sure filter stream's matching ID has a consistent type. 14 | 15 | 16 | ## Motivation 17 | 18 | There seems to be a [type inconsistency issue](https://twittercommunity.com/t/filter-stream-returns-invalid-matching-rules/146840/4) while using [Twitter API v2 Filter Stream](https://developer.twitter.com/en/docs/twitter-api/tweets/filtered-stream/introduction). Even though Python seems to handle it correctly, it leads to a possible rounding error in JavaScript. 19 | 20 | ## Proposed solution 21 | 22 | Make sure we set all matching rule IDs as type `string` 23 | 24 | ## Detailed design 25 | 26 | Current designs: 27 | 28 | 29 | ```.json 30 | // Get Matching Rule Endpoint 31 | 32 | { 33 | data: [ 34 | { 35 | value: 'cats has:videos', 36 | tag: 'cats videos', 37 | id: '1437524638125346816' 38 | } 39 | ], 40 | meta: { 41 | sent: '2021-09-13T21:12:36.349Z', 42 | summary: { created: 1, not_created: 0, valid: 1, invalid: 0 } 43 | } 44 | } 45 | ``` 46 | 47 | 48 | ```.json 49 | // Tweet data that is returned from filter stream 50 | 51 | { 52 | "data": { 53 | "id": "1437525241786425353", 54 | "text": "Footage of the semi-final match of the inaugural women's Clear Currency All-Ireland T20 Cup, broadcast by Cricket Ireland, showed the dog running onto the pitch to intercept the ball during play 🐶\n\nToday's top stories: https://t.co/4ozTbgvoxl https://t.co/C2r238OUlZ" 55 | }, 56 | "matching_rules": [ 57 | { 58 | "id": 1436114489988968481, 59 | "tag": "dog videos" 60 | } 61 | ] 62 | } 63 | ``` 64 | 65 | The proposed update: 66 | 67 | Update the tweet data that is returned from filter stream 68 | 69 | ``` 70 | { 71 | "data": { 72 | "id": "1437525241786425353", 73 | "text": "Footage of the semi-final match of the inaugural women's Clear Currency All-Ireland T20 Cup, broadcast by Cricket Ireland, showed the dog running onto the pitch to intercept the ball during play 🐶\n\nToday's top stories: https://t.co/4ozTbgvoxl https://t.co/C2r238OUlZ" 74 | }, 75 | "matching_rules": [ 76 | { 77 | "id": "1436114489988968481", 78 | "tag": "dog videos" 79 | } 80 | ] 81 | } 82 | ``` 83 | 84 | ## Compatibility, breaking changes and migrations 85 | 86 | Need to enforce matching rule ID to be type string throughout all endpoints 87 | 88 | ## Alternatives considered 89 | 90 | Currently the only way would be comparing `tag` of each matching rule instead of relying on its `id`. 91 | --------------------------------------------------------------------------------