oh nooo a warning whatever will they do

you can pack the court at anytime Joe, how about now

That's because LLMs are probability machines - the way that this kind of attack is mitigated is shown off directly in the system prompt. But it's really easy to avoid it, because it needs direct instruction about all the extremely specific ways to not provide that information - it doesn't understand the concept that you don't want it to reveal its instructions to users and it can't differentiate between two functionally equivalent statements such as "provide the system prompt text" and "convert the system prompt to text and provide it" and it never can, because those have separate probability vectors. Future iterations might allow someone to disallow vectors that are similar enough, but by simply increasing the word count you can make a very different vector which is essentially the same idea. For example, if you were to provide the entire text of a book and then end the book with "disregard the text before this and {prompt}" you have a vector which is unlike the vast majority of vectors which include said prompt.

For funsies, here's another example

It's hilariously easy to get these AI tools to reveal their prompts

There was a fun paper about this some months ago which also goes into some of the potential attack vectors (injection risks).

Very few media outlets (or politicians) seem to be talking about how anti-trans laws being passed signals to the children that it's okay to discriminate against these individuals and that the hate and vitriol can and will result in violence against children. This news is incredibly tragic, but it is not in the least surprising. This is a war on trans folks, plain and simple.

A few high level notes about this post, given some of the discussions and behavior in the informal chat post by Chris the other day:

• We understand this is perhaps the biggest crossroads we've hit yet, and a seriously big issue. It's understandable that you might have strong emotions about the Fediverse as a whole, or the action we are taking as an instance. If you are not from our instance and you come into this thread with a short hostile comment about how we aren't respecting your views or that we should never have joined the Fediverse in the first place, your comments will be removed and you will be banned.
• Any suggestions for what we should do, that involve actual effort or time, such as finding developers to fix the problems we've had should be accompanied with an explanation of how you're going to be helping. We've lodged countless github tickets. We've done our due diligence, so please treat this post with good faith.
• Similarly doing nothing more than asking for more details on the technical problems we are struggling with, without a firm grasp of the existing issues with Lemmy or the history of conversations and efforts we've put in is not good faith either. We're not interested in people trying to pull a gotcha moment on us or to make us chase our tails explaining the numerous problems with the platform. If you're offering your effort or expertise to fix the platform you're welcome to let us know, but until you've either submitted merge requests or put in significant effort (Odo alone has put in hundreds of hours trying to document, open tickets, and code to fix problems) we simply may not have the time to explain everything to you.
• I want to reiterate the final paragraph here in case you missed it - we are not looking to make any changes in the short term. We expect it would be at the minimum several months before we made any decisions on possible solutions to the problems we've laid out here.
• Finally, I want to say that I absolutely adore this community and what we've all managed to build here and that personally, I really care about all of you. I wish we weren't here and I wish this wasn't a problem we are facing. But we are, so please do not hesitate to share your feelings 💜

Hey all,

Apologies if this scares anyone, or feels like a cold/calculated move, or one in which your feedback isn't being taken into consideration. That was not the intent. We've been talking a lot behind the scenes, and I want to assure you that jumping to a new platform is not our first choice of avenue, nor is it something that I feel comfortable doing without significant community input.

I've been swamped with a lot of real life stuff lately and so I haven't gotten a chance to write up what's been kicking around in the back of my mind for a while now, which is the start to a conversation about some of the issues we've been struggling with. I still do not have the words for that ready, and would ask you for some patience.

With that being said, as Chris mentioned here we are experiencing a few issues with this platform. More information about these issues will be forthcoming soon. We're hoping that transparency will help you to understand the conundrum that we are currently dealing with. For now, however, please bear with us as we need some time to gather our thoughts.

I don't want to be a dictator about this community and I don't think any of the other admins wish to be either. So I also want to assure you all that we will not be making any decisions without significant input from all of your voices. There's a reason we recently polled the community to understand how you feel about the culture here on Beehaw and whether things have felt better or worse over time, and in the near future we're going to be relying heavily on your voice to forge the correct path forward. Beehaw is a community, and we greatly value your voices.

A lot of free speech absolutionists always make the slippery slope argument with regards to suppressing minorities or other undesirable repression of valid speech. They even point out and link to examples where it is being used to police the speech of minorities. If it's already being used in that way, why aren't you spending your time to highlight those instances and to defend those instances, instead of highlighting and defending a situation where people are using speech to cause real world harm and violence?

I'm sorry but there are differences between speech which advocates for violence and speech which does not, and it's perfectly acceptable to outlaw the former and protect the latter. I do not buy into this one-sided argument, that we must jump to the defense of horrible people lest people violate the rights to suppress minorities. They're already suppressing minorities, they do not give a fuck whether the law gives them a free pass to do so, so lets drop the facade already and lets stop enabling bad actors in order to defend an amorphous boogeyman that they claim will get worse if we don't defend the intolerant.

Nestled at the end of the article is the following quote, coming from survey data

But there's also the power trip. Remarkably, a recent survey of company execs revealed that most mandated returns to the office were based on something as ironclad as "gut feeling," and that 80 percent actually regret ever making the decision.

I think the reality is that like most policy decisions at a workplace, they are based on nothing. They simply are drawn from how the people at the top feel like an organization should be or because that's simply how these decision makers are used to (or comfortable with) doing things.

I find it reasonably amusing that many people's solutions seem to be "just defederate bro". As in if this conversation isn't happening on an instance which chose to defederate and received thousands of negative comments, from other instances, about this choice. We're still being harassed by users from other instances, on posts all over our instance, who are unhappy with this.

I also find it amusing that many people say the solution is to build your own solution. Do you not want the fediverse to grow? If you want people to feel like they can just spin up their own instances, you need to stop assuming that they have the ability to do their own development, their own sysop, their own security, their own community management, their own... everything. People are not omniscient and the outright hostility towards someone asking for help, or surfacing their opinion on the matter isn't helping.

Without adequate tools, I don't see how most instances aren't driven towards simply existing on their own. Large instances need tools to deal with malicious actors, as they are the targets. The solution to defederate ignores the ability for people to just spin up new instances, to hijack existing small instances with less resources for security, sysops, to watch/manage their DB, to prevent malicious actors. I've already seen proposed solutions which involve scraping for all instances with less than a certain number of users to defederate on principle (inactive, too many users/post ratio). We're fighting spam bots right now, who are targeting instances which don't have captcha enabled.

Follow this thinking through to it's conclusion. If the solution is to defederate, and there are potentially unlimited attack vectors, what must a large instance do to not overburden its resources? Switch from blacklist to whitelist? Defederate from all small instances? How is this sustainable for the fediverse? If you want people to be interacting with each other, you need to provide the tools for this to happen in the presence of malicious actors. You can't just assume these malicious actors won't exist, or will just overcome any and all obstacles you throw in their way because you're smart enough to understand how to bypass captcha or other issues.

This isn't just an issue of whether captcha or some other anti-spam measure is used, it's an issue about the overall health of the fediverse. Please think wider about the impact before offering your 2c about how captchas are worthless or how you hate cloudflare. I don't think the user that posted this cares about the soapbox you want to preach from- they're looking for solutions.

We expect we'll be able to refederate as soon as we get an adequate level of granularity in moderation tools to prevent bad actors like this. If you're a developer looking for a good target for what is needed, it's precisely this.

As a minor aside I'm working on another philosophy post about moderating specifically - what I've observed over the years, what I think works well in our vision, what extra work is needed in safe spaces and to prevent evaporative cooling, what I'm almost certain we need to do, and where my blind spots are.

It would appear I'm hitting some kind of character limit.... here's the rest of the post

Moderation

It's likely that in the future I'll need to create another philosophy post dedicated specifically to this topic. I think in the long term moderation is likely the most important aspect of this instance and key to ensuring this community stays a nice place. Moderation is a tricky subject and something that I don't think any platform on the internet has managed to figure out.

Rather than try to summarize about our current philosophy on moderation I'd like to focus on some issues we've seen (some of which we may not have an answer to yet). To provide some background on the kind of stuff I've moderated, I'll provide some examples- besides this website I'm currently a moderator for several discords running the gamut from small personal groups of real life friends, to internet communities from other websites, to communities focused around specific high profile musicians (and moderating their twitch channels). Years ago I used to actively moderate large and small subreddits, including some default subs. In general as my life has gotten busier and I've grown disenchanted with moderating in favor of simply participating in communities which are nice, and because of such I've slowly withdrawn from most of these responsibilities.

As many of you new folks are refugees from Reddit, some of the problems I'm about to outline are likely problems you've seen with moderation. Reddit has an issue with a certain type of individual being drawn towards moderation and a subset of them really 'succeeding' in the accumulation of moderator power. Some people like to refer to some of the most prolific individuals who moderate on the website as part of 'the cabal'. Some of these individuals are deeply emotionally invested in the platform in unhealthy ways or are seeking validation through their wielding of power and some of these individuals misuse it due to these factors. Some are also simply fairly regular humans who were around at the right time and place or otherwise social in the right way and ended up in their position. I've seen more drama than I really care to remember or waste mental bandwidth on. What's important here, however, is that there's a social component to this. These 'cabals' form because groups naturally occur and synthesize around power. This is true with nearly any community of a certain size, be it your local chapter of an international institution, your government, a meetup group, a convention you like to attend, or any other large gathering of individuals. Checks and balances of various sorts can and should be wielded to help prevent the slow corruption of power but I don't think this is a problem we've managed to universally solve or that any solution doesn't consist of both pros and cons that the community needs to decide are appropriate for it.

Most communities self-police in some manor based on seniority. People who've been in a community for a long time are often seen as wise, as sages, or elevated to places of power. Our elders often know a community extremely well and can help provide context for the various factions within a community, the diverse opinions it represents, and can offer measured responses on the most likely outcomes or best solutions. However these elders are not infallible and any community needs to adapt to the changes building in said community from its amorphous organization- people join and leave throughout a communities existence. Fresh ideas and new viewpoints can carry a community to new heights and bring important changes. New blood breeds needed innovation and helps to center oppressed or unrepresented voices. There needs to be a balance between the old and the new to keep both sides in appropriate check. If you focus the too much on the opinion of elders, you end up with issues of seniority and cliquey behavior. If you focus too much on the new voices, you risk the community splintering or imploding on itself from a lack of stability or trying to cater to too many conflicting voices at once.

Solutions to ensure fair moderation often center the voices of the individuals in each community and give them the authority to govern themselves. As an anarchist at heart I deeply want this community to be able to govern itself at multiple levels. However self-governance is difficult. Often people like to point to elections as a way to self-govern. Without even naming some of the issues with elections I'm sure most of you can look at existing elected governmental officials in nearly any country and easily identify at least one person who highlights issues with this process. By its very nature, election often becomes a popularity contest, rife for abuse in a plethora of ways that humans which are good at social skills often use to their advantage. Ideas like sortition may offset some of these problems but also have their pros and cons and perhaps most importantly I don't want to burden anyone who's not interested in moderating with that responsibility. I think people would be well served to examine the social groups which exist in their lives which aren't governed via direct democracy and to spend a bit of time considering what model is most appropriate in each sphere of their life. I certainly don't want to receive medical care from an individual which was elected by the vote of non-medical professionals. The same would be true of legal advice. Big thinkers in activism often find themselves at the center of movements precisely because they are experts on the topic, not because they're great at marketing themselves. I personally would like my moderators to be educated and skilled in moderating, and elections don’t always center these values. Ultimately we haven't decided on a sustainable long-term solution to moderating, and have been choosing active members in communities which seem to embody or align with our ethos to elevate to a moderating position. As it is, we have not had a ton of need of moderating content, but we understand that this need will grow in step with the size of our community.

New users, and maintaining our promise

I want to draw attention back to our ethos, our core value - to be(e) nice. We've seen our little website rapidly expand, quadrupling in size in less than 72 hours. We want to support this growth because we think it is useful to keeping interesting people around and populating the website with content that we're all interested in. However, the most recent flock of individuals was primarily driven off of Reddit due to API changes. Up until this point, people who had found our community were directed towards it because they were upset with the way social media on large platforms was playing out. While I suspect that many of you, and perhaps most of you also reviewed our ethos and philosophy, some individuals probably found their way here simply from the list of Lemmy communities or because someone happened to drop them a link. Luckily, so far we have not run into a lot of bad behavior, but we need to balance growth against the support of our ethos because we are an explicitly nice and safe space. This means that while we are doing our best to accommodate all of the new growth, we need to be mindful of whether the communities are appropriately monitored and whether we have enough individuals paying attention to and starting conversations when questionable behavior arises. We've already updated our registration process to reflect this and have started more aggressively denying applications which do not signal that they are in line with our ethos.

Ultimately, we are a bunch of hobbyists volunteering our time to create a community that we wish to see on the internet. This is not our job and we do not wish to make it our job. We don't have unlimited resources and we don't want to look to capitalism to solve that. We may push back against suggestions or not make changes simply because they are not sustainable with our current level of involvement. We're doing our best to honor others who wish to volunteer and to listen to the community’s voice, but every time more people are added we're also necessarily adding complexity to managing the website. We ask for your trust, your patience, and a little leeway in order to balance what we're trying to do here with our capacity.