Karl Anderson › Comments

No, they will normally not need to re-authorize. If the token is revoked, however, they will, and your application needs to be able to handle this (by allowing them to do so). In general, this is true for any SSO system. A twitter user can explicitly revoke an application's token at the provided page. - Karl Anderson

plexistar recommended - Karl Anderson

This is implementation-specific - you'll have to see what the Delicious docs say about the token. It may expire, have limited uses, or have side effects when used. Most OAuth implementations will probably expire their tokens at some point to reduce the number of valid tokens they have to keep track of. In general, user-agent help should make this less of an issue for SSO authentication systems - when the user shows up without a valid token, the browser is redirected to the authenticator, which looks at stored credentials on the browser (usually cookies) and redirects the user back with a new token, without any user interaction. This can be more complex for OAuth than for OpenID, since it might not be appropriate to issue a new token if it does more than authenticate. And since the authentication/authorization process is implementation specific, you need to be able to enter new credentials unless you know that the token will be valid. - Karl Anderson

I did this by just writing the output directly from my reducer method to S3, using an S3 toolkit. Since I was running on EC2, this was quick and free. In general, you want Hadoop to handle your input and output as much as possible, for cleaner mappers and reducers; and, of course, you want to write to S3 at the very end of your pipeline, to let Hadoop's code moving do it's job over HDFS. In any case, I recommend doing all of your data partitioning, and writing entire output sets to S3 in a final reduce task, one set per S3 file. This puts as little writer logic in your code as possible. This paid off for me because I ended up with a minimal Hadoop S3 toolkit which I used for several task flows. I needed to write to S3 in my reducer code because the S3/S3n filesystems weren't mature; they might work better now. - Karl Anderson

cheap cotton/wool army surplus socks - Karl Anderson

various 5.11 style clothes cheaper than BDUs - Karl Anderson

makes the color of your computer's display adapt to the time of day, warm at night and like sunlight during the day. - Karl Anderson

Hashing or randomizing identifiers or other URL components can be a good practice when you don't want your URLs to be traversable. This is not security, but it will discourage the use (or abuse) of your server resources by crawlers, and can help you to identify when it does happen. In general, you don't want to expose application state, such as which IDs will be allocated in the future, since it may allow an attacker to use a prediction in ways that you didn't forsee. For example, BIND's sequential transaction IDs were a security flaw. If you do want to encourage crawling or other traversal, a more rigorous way would be to provide links, rather than by providing an implementation detail which may change in the future. Using sequential integers as IDs can make many things cheaper on your end, and might be a resonable tradeoff to make. - Karl Anderson

Within Twisted, you basically want a wrapper around a function which returns a Deferred, such as the Twisted DB layer. However, you can't busy-wait, since that's using up your reactor cycles, and checking for a task to complete using the Twisted non-blocking wait is probably inefficient. Will inlineCallbacks or deferredGenerator solve your problem? They require a modern Twisted. See the twistedmatrix docs. def thingummy(): thing = yield makeSomeRequestResultingInDeferred() print thing #the result! hoorj! thingummy = inlineCallbacks(thingummy) Another option would be to have two methods which execute the same SQL template, one which uses runInteraction, which blocks, and one which uses runQuery, which returns a Deferred, but that would involve more code paths which do the same thing. - Karl Anderson

Just write to the file. You have a Python interpreter. - Karl Anderson

various config including x11 - Karl Anderson

includes x11 locales, which enables more prefs - Karl Anderson

You'll be signing requests. Any OpenID or OAuth library will have code which does that - if not a separate library, something you can rip out, even though you're not using the protocol flow. Alternately, there's demo code for Amazon AWS clients in their developer docs which also signs requests with API keys and timestamps. It's the same process on the client or server side - comparing a hash with the original - so you'll probably just have to translate their request model to whatever your framework uses. Neither of these are exactly what you're asking for, but it's a start. - Karl Anderson

For searches, at least, they'll throw up a captcha if anyone from your IP is acting like a robot, and you probably can't control who shares the IP you're behind. So it might not depend on anything you do. - Karl Anderson

<link rel="alternate" href="http://publisher.example.com/happyca... - Karl Anderson

Hookah The HTTP event engine Current Features * Interface is 100% HTTP (use easily from any language) * Dispatches POST requests (webhooks) asynchronously with retry * Provides publish/subscribe interface using PubSubHubbub protocol (experimental) * Provides "Twitter Stream API"-style long-polling interface for topics (super experimental) About Hookah was originally created to ease the implementation of webhooks in your web systems. While webhooks are still at the core, it's becoming a scalable HTTP event engine with HTTP pubsub and long-polling event streaming. And of course, webhooks. Any system with webhooks or looking to implement webhooks will benefit from Hookah. - Karl Anderson

As others mentioned, you should be using Trial for unit tests in Twisted. You also should be unit testing from the bottom up - that's what the "unit" in unit testing implies. Test your data and logic before you test your interface. For a HTTP interface, you should be calling processGET, processPOST, etc with a mock request, but you should only be doing this after you've tested what these methods are calling. Each test should assume that the units tested elsewhere are working as designed. If you're speaking HTTP, or you need a running server or other state, you're probably making higher level tests such as functional or integration tests. This isn't a bad thing, but you might want to rephrase your question. - Karl Anderson

Iterative algorithms are fine with a MapReduce framework, for example "run this job on the previous job results until a condition is met or we decide to give up". There are job control schemes for Hadoop which abstract this away behind a query language. What the map-reduce paradigm doesn't do is communicate between nodes - no "start reducing when enough results are found among all mappers". So yes, no overlapping or skipping one un-needed mapper because another found what was needed. - Karl Anderson

Always Be Coding. Practice will always be the most important thing you can do. Read the design patterns books others have recommended. Then code some more. Learn a MVC framework. Rails is the currently popular one. Code something with storage requirements which can be solved with data structure manipulation behind the scenes. Alternately, play with Hadoop (Pig, Cascading, etc.) or another dataflow abstraction. Study math, law, or other logical fields. Play sokoban, it's all about seeing parity, cycles, and other higher order structures. Or play go or chess. Then code something. - Karl Anderson

mitering templates - Karl Anderson

We had these at school too. "lsd" printed an ASCII-art molecular diagram for people who mistyped "ls". "win" would reply "you win!" for people who forgot what kind of box they were on. - Karl Anderson

If you're only using one token which is given by the server on the initial authentication, it can be used for any request if it's intercepted. Your only defense is the expiration time. Beyond that, it depends on what your implementation options are. A more secure system is to add a timestamp (and possibly a nonce) to each request, sign that, and include that with each request. It requires that the client handles the authentication credentials, knows the signing implementation, and signs each request. You could alternately have the server authenticate with each request (which could be done with OpenID) or hand out a number of tokens and re-authenticate when more are needed (which could be done with OAuth). If the client can store credentials, these can be invisible to the user. These are more complex, requiring an encrypted transport such as SSL for some of the interactions, and a client which can speak HTTP redirects and handle cookies or other stored state. The client wouldn't have... - Karl Anderson

The short answer is yes, in practice, by choosing a reasonable OpenID provider. The long answer is that this is a question about your OpenID provider, not OpenID itself. OpenID only handles authentication. Session support (and related concepts such as logging in or out) is outside of the OpenID spec. Your OpenID provider may keep a session for you (probably with a browser cookie). Any reasonable provider will not do this unless you indicate ("remember me when logging into stackoverflow.com"). The OpenID consumer (such as StackOverflow) may keep a session for you as well. There is nothing you can do about this, but this is true for any authentication scheme, whether 3rd party or direct. Because the 3rd-party aspects of OpenID are well designed, it is easy to avoid logging in in practice, by choosing a reasonable provider, or better yet, by using a HTML page as your claimed identity, which delegates to a provider which you can change if you decide to switch. - Karl Anderson

The client has been sent to an authorization page on server B via a HTTP redirect. Since you don't know anything about server B, you don't know what that authorization (and/or authentication) involves; it's out of scope for OAuth. You don't know that you have the user's credentials for sever B, since you don't know what they are. Normally, the client is the user's browser of choice, and server B may choose to accept authentication and/or authorization credentials which it has stored on the client (usually with a cookie), without user interaction. However, this is also out of your control; you can't stop server B from requiring user interaction. If you only need authentication, OpenID is more likely to allow this, since there ususally aren't any ID choices, but it still is not certain. You may be able to store an access token for later renewal and use without re-authorizing at all. This is also up to server B, and you can't rely on this for a generic server. - Karl Anderson

Why use the proposed OpenID OAuth Extension over another OAuth-based protocol? Discovery doesn't seem to be a feature. Although the consumer only needs a claimed identifier to start the authentication process, any authorization will still require that the consumer knows the access token URL, signature methods, and have a shared key/secret with the provider. One benefit is that there's a specified way to ask for specific authorizations, the openid.oauth.scope argument to the authentication request. And one specific benefit for this is that authentication only - for the benefit of the consumer only, with no verifiable token - is defined for free and can be performed without worrying about keeping track of outstanding tokens or the resources they might require. Are there examples of alternative ways in use to specify the scopes to be requested, perhaps with something in OpenID discovery? Or could this be handled effectively with some kind of REST navigation before the OAuth process,... - Karl Anderson

No, but you did opt in to hear whatever the heck was on their minds - and the unsub link is right there :) - Karl Anderson

So long as your users/clients can accept them, you can sign an SSL certificate for free. I don't think this gets around the problem of exchanging keys beforehand, but it reduces the exposure. - Karl Anderson

Rather than passing authentication tokens, Carmen could use OpenID to authenticate Alice through Bob. This wouldn't involve Alice giving Carmen any credentials. It could involve another user interaction, but user-agent assistance (signed cookies or client certs) should reduce this (I generally have to interact with my OpenID provider the first time I authenticate for a consumer). This assumes that Bob and Carmen can communicate each time authentication is needed. - Karl Anderson

Not much information here about what your security or implementation needs are. The quick answers are Basic or Digest over SSL, or signed requests. Are there reasons not to use these? Signing requests typically adds a timestamp and/or a nonce, so any request can be authenticated. See the Amazon AWS authentication documentation for a description and libraries. - Karl Anderson

You can securely authenticate without needing to implement protection against eavesdropping. For example, you can prevent others from sending requests, even though they can read the contents of your requests. If you need to protect against eavesdropping, I'd recommend just going somewhere where you can use SSL. If you just need simple authentication without real security, your provider will probably support HTTP Basic. This (along with a good design which limits capabilities, and backups ;) is a reasonable interim solution while you worry about other problems. For authenticating your identity, OpenID can't be replayed. Each authentication sequence is signed. However, OpenID by itself only lets you establish your identity with the server. It won't let you sign or otherwise authenticate a request. OAuth would, but it requires transport encryption for part of the protocol. You could sign each request with a shared secret. This would prevent an attacker from submitting or replaying a... - Karl Anderson